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1893 — L.A. Sherman: The Analytics of Literature 

Down through the centuries, many had written about the dif- 
ferences between an “ornate” and “plain” style in English. 

In 1880, a professor of English Literature at the University of 
Nebraska, Lucius Adelno Sherman, began to teach literature 
from a historical and statistical point of view. He gave the 
world a new method of literary criticism. 

Sherman, like other teachers of his time, saw literature as a 
method for the moral and spiritual edification of citizens. He 
was one of the first to recruit science for this task and to ad- 
vocate an “objective” approach to literature. In 1893, he pub- 
lished Analytics of Literature : A Manual for the Objective 
Study of English Prose and Poetry. 

He found his approach was highly effective in giving his stu- 
dents an appreciation for great literature: 

Students apparently without taste for reading, or ca- 
pacity to discern common literary excellencies, were 
enabled to appreciate and enjoy poetry as well as the 
best. Bright scholars were also in their way benefited 
not less than the undiscerning. Things vague were 
made definite. Grounds of judgment before indetermi- 
nate or hidden were made plain. Criticism was ren- 
dered confident; and no little enthusiasm was aroused 
(p. xi). 

In general, this method, if tried intelligently and fairly, 
will discover to those who suppose they have no taste 
for the best literature that they have such taste; and it 
will make those who have never found anything in po- 
etry both feel and know something of its power (p. xii). 

In defending his method against critics, he wrote: 

There is a very natural antipathy to treating aesthetics 
by scientific methods. Yet there is in the nature of 
things no reason why we may not as well analyze the 
tissues of human speech and though as the tissues of 
the human body. Within a generation science has 
been broadened by the use of imagination, and there 
is no good reason why aesthetics in turn should not 
have the material aid of facts and statistics (p. xiii). 

The proof, he claimed, is in the results. His method immedi- 
ately engages students with the text on a practical level they 
can understand. A more refined appreciation of the content- 
grows out of the familiarity with the form and structure of the 
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text: 

Sherman's work makes modern use of statistics, charts, and 
graphs. Most notable are his findings about the streamlining 
of language. 

In comparing the older prose writers with the then-current 
writers such as Macaulay and Ralph Waldo Emerson, 

Sherman noticed a progressive shortening of sentences over 
time. 

He decided to look at this statistically and began by counting 
average sentence length per 100 periods. In his book he 
showed how sentence-length averages shortened over time: 

• Pre-Elizabethan times: 50 words per sentence 

• Elizabethan times: 45 words per sentence 

• Victorian times: 29 words per sentence 

• Sherman’s time: 23 words per sentence. 

In our time, the average is down to 20 words per sentence. 

Sherman' s work set the agenda for a century of research in 
reading. It proposed the following: 

• Literature is a subject for statistical analysis. 

• Shorter sentences and concrete terms increase read- 
ability. 

• Spoken language is more efficient than written lan- 
guage. 

• Over time, written language becomes more efficient 
by becoming more like spoken language. 

Sherman also showed how individual writers are remarkably 
consistent in their average sentence lengths. This consistency 
was to become the basis for the validity of using samples of a 
text rather than the whole thing for readability prediction. 

Another of Sherman' s discoveries was that over time sen- 
tences not only became shorter but also simpler and less ab- 
stract. He believed this process was due to the influence of the 
spoken language on written English. He wrote: 

Literary English, in short, will follow the forms of 
the standard spoken English from which it comes. No 
man should talk worse than he writes, no man writes 
better than he should talk. . . . The oral sentence is 
clearest because it is the product of millions of daily 
efforts to be clear and strong. It represents the work 
of the race for thousands of years in perfecting an ef- 
fective instrument of communication (p. 312). 
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Linguistic research later confirmed Sherman’s view of the re- 
lationship between spoken and written language. 




Sherman showed the importance of 
average sentence length and the re- 
lationship between spoken and 
written English. 



Sherman’s most important point 
was the need to involve the 
reader. He wrote: 

The universally best style is 
not a thing of form merely, but 
must regard the expectations 
of the reader as to the spirit 
and occasion of what is writ- 
ten. It is not addressed to the 
learned, but to all minds. 
Avoiding book-words, it will 
use only the standard terms 
and expressions of common 
life. . . It will not run in long 
and involved sentences that 
cannot readily be understood. 
Correct in all respects, it will 
not be stiff; familiar, but safely 
beyond all associations of vul- 
garity (p. 327). 

— WHD 
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1921 — E. L. Thorndike: The Teachers’ Word Book 

During the 1920s, two major trends stimulated a new interest 
in readability: 

1. A changing school population, especially an increase in 
“first generation” secondary school students, the children 
of immigrants. Teachers reported that these students 
found textbooks too difficult. 

2. The growing use of scientific tools for studying and objec- 
tively measuring educational problems. 

One such tool, Thorndike’s Teacher’s Word Book, which 
came out in 1921, was the first extensive listing of words in 
English by frequency. It provided teachers with an objective 
means for measuring the difficulty of words and texts. It laid 
the foundation for almost all the research on readability that 
would follow. It was also the basis for the first readability 
formulas. 

Its author, psychologist Edward L. Thorndike of Columbia 
University, noticed that teachers of languages in Germany and 
Russia were using word counts to match texts with students. 
The more frequent a word is used, they found, the more famil- 
iar it is and the easier to use. As we learn and grow, our vo- 
cabulary grows as does our ability to master longer and more 
complex sentences. How much that continues to grow de- 
pends on how much reading is done throughout life. 

Around 1911, Thorndike began to count the frequency of 
words in English texts. In 1921, he published The Teacher’s 
Word Book , which listed 10,000 words by frequency of use. 

In 1932, he followed up with A Teacher’s Word Book of 
20,000 Words, and in 1944 with Irving Lorge, A Teacher’s 
Word Book of 30, 000 Words. 

A vocabulary test on the meaning of words is the strongest 
predictor of verbal and abstract intellectual development. The 
knowledge of words has always been a strong measure of a 
reader’ s development, reading comprehension, and verbal in- 
telligence. Chall and Dale wrote in 1995, “It is no accident 
that vocabulary is also a strong predictor of text difficulty.” 

It happens that the first words we learn are the simplest and 
shortest. These first, easy words are also the words we use 
most frequently. Most people do not realize the extent of this 
frequency. Twenty-five percent of the 67,200 words used in 
the 24 life stories written by university freshmen consisted of 
these ten words: the, I, and, to, was, my, in, of, a, and it. The 
first 100 most frequent words make up almost half of all writ- 
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ten material. The first 300 words make up about 65 percent of 
it. 

Educators, publishers, and teachers still use word-frequency 
lists to evaluate reading materials for schools. After 
Thorndike, there was extensive research on vocabulary. The 
high mark came in 1949 with Human Behavior and The Prin- 
ciple of Least Effort by Harvard’s George Kingsley Zipf. 

Zipf used a statistical analysis of language to show how the 
principle of least effort works in human speech. Zipf showed 
that, in many languages, there is a mathematical relationship 
between the hard and easy words, now called Zipf s curve. 
This notion of saving energy is a central feature of language 
and is one of the principle bases of research on the frequency 
of words. 

In 1968, psychologist George Klare wrote, “Not only do hu- 
mans tend to used some words much more often than others, 
they recognize more frequent words more rapidly than less 
frequent, prefer them, and understand and learn them more 
readily. It is not surprising, therefore, that this variable has 
such a central role in the measurement of readability.” 

— WHD 
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1923 — The Lively and Pressey Measuring Method 

Introduction 

Bertha A. Lively and Sidney L. Pressey of Ohio State Univer- 
sity were concerned with the practical problem of selecting 
science textbooks for junior high school. Books at that time 
were so overlaid with technical words that teachers spent all 
class time teaching vocabulary. 

In 1923, they published their study, “A Method for Measuring 
the ‘Vocabulary Burden’ of Textbooks” in the journal Educa- 
tional Administration and Supervision .' 

They argued that it would be helpful to have a way to measure 
and reduce the “vocabulary burden” of textbooks. 

Their study tested three different methods for measuring the 
vocabulary load of a thousand words of text: 

1 . The first method used the number of different words (the 
vocabulary range). 

2. The second method used the number of “zero-index 
words,” words notin The Teacher’s Word Book, the 
Thorndike list of 10,000 words. 

3. The third method used the median of the index numbers 
of the words taken from the same Thorndike list of 10,000 
words. 

They tested the three methods on 15 textbooks of different 
difficulties, along with one newspaper. The low end included 
a second and a fourth-grade reader and Stevenson’s Kid- 
napped. The high end included a college physics textbook and 
an elementary chemistry textbook. 

They found that the median index number was the best indica- 
tor of the vocabulary burden of these reading materials: the 
higher the index number, the easier the vocabulary; the lower 
the index, the harder the vocabulary. 

The Lively-Pressey study demonstrated the effectiveness of a 
statistical approach for predicting text difficulty. It had a great 
influence on the readability formulas that would follow and 
also use the Thorndike word list. As the authors announced, 
“The fundamental value of Thorndike's contribution is obvi- 
ous; the ‘Word Book’ has opened up a whole new field for 
investigation.” 

— WHD 



1 Vol. 9, No. 7 (October 1923), pp. 389-398. 
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INCLUDING TEACHER TRAINING 



Vol. IX October, 1923 No. 7 

A Method for Measuring the "Vocabulary 
Burden" of Textbooks 

BERTHA A. LIVELY AND S. L. PRESSEY 
Ohio State University 

I. Need for a Systematic Method for Investigating 
Vocabulary Burden 

T he present study was begun as a result of a minor investi- 
gation regarding the number of technical words in a cer- 
tain junior high school science book. The study revealed an 
astounding number of technical terms — a number so large (as 
testified by teachers using this book) that the course often be- 
came quite as much a study of scientific vocabulary as of sci- 
entific facts. This investigation brought out in striking fashion 
the importance of the question as to comparative vocabulary 
burden, in public school textbooks. The problem is perhaps 
most acute in connection with junior high school science 
books. But it is also an important problem in reading; some 
method for measuring vocabulary difficulty in supplementary 
reading material should be decidedly worth while. The present 
paper presents an effort to develop a method capable of deal- 
ing with these problems, together with results of application 
of this method to certain representative textbooks from second 
grade readers to a medical school physiology. 1 

II. Method and Material 

The method finally adopted, after an extended series of pre- 
liminary investigations which need not be presented here, may 
be briefly described. Two questions were involved in the 
elaboration of procedure; (a) How many words must be in- 
cluded in any sampling from a textbook in order to obtain a reli- 
able indication regarding vocabulary — and how should these 
words be selected? ( b ) How can the difficulty of the words in 
this sampling be best measured? 



1 The writers wish to acknowledge their obligations to the Graduate 
Council of the University for funds to assist in the clerical labor. 
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The method of vocabulary sampling finally adopted dealt with 
thousand word units obtained from a systematic sampling 
throughout the text. First of all, the investigator noted the 
number of pages in the book and the approximate number of 
words per line; he then estimated the number of pages which 
should be sampled, taking one line per page, in order to cover 
1000 words, and chose pages so that the sampling would be 
evenly distributed throughout the book. Thus if the book con- 
tained approximately 500 pages, and there were about 10 
words to the line, a line on each fifth page throughout the 
book would make up about a thousand words. The investiga- 
tor then went through the book, counting up the number of 
words found on the third line of each fifth page until exactly 
1000 words were obtained. The third line was used as conven- 
iently found on a page. If the book were shorter, every other 
page might be taken, if much longer, every tenth; pages were 
chosen simply to give a systematic sampling throughout the 
book. The lines chosen through to the thousandth word were 
now gone over, and all the different words found in this thou- 
sand listed and alphabetized. 

Once the thousand-word count was made, the total number of 
different words per thousand was first noted. This gave what 
has been called vocabulary range. Next, these words were 
looked up in the Thorndike "Word Book" 2 and the index 
number for each word was found. The number of words was 
now counted, in the thousand-word sampling, not appearing 
among the most common 10,000 words; these words were 
listed as zero value words and may be taken to indicate the 
size of the technical vocabulary. The weighted median index 
number was finally calculated. This is simply the median in- 
dex number with zero value words counted twice. Evidently 
the higher the median index number the easier the vocabulary. 

Sixteen different types of reading matter were thus studied; 
Three second-grade readers (Jones, Aldine, Horace Mann), 
three fourth-grade readers (Jones, Aldine, Horace Mann), Ste- 
venson's "Kidnapped/” Thackeray's "Vanity Fair” the Colum- 

2 

Thorndike, E. L.: "The Teacher's Word Book." Teacher's College, Bureau of 
Publications, Columbia University, New York City. This book lists the 10,000 
most common words of the English language, as determined on the basis of an 
elaborate investigation by Professor Thorndike. In this "Word Book" each word 
is followed by an index number indicative of its commonness. Thus such a 
common word as "and" has an index number of 210; a relatively uncommon 
word like "atom" has an index number of 4; still more rare words such as "neo- 
lithic' 1 do not appear in the word book at all — those are listed as zero value 
words. Words with credit numbers of 49 or over occur in the first 1000 words, 
in frequency. Words with index numbers of 10 or over occur in the first 5000 
words — and so on; for a more detailed statement, the reader is referred to the 
"Word Book" itself. 



8 





1923 — The Lively and Pressey Measuring Method 



bus Dispatch (as a sampling of newspaper vocabulary — only 
the first page was taken), Muzzey's "American History” 
Clark's "General Science" and "Introduction to Science” (as 
representative of Junior High School books in science), 
Hunter's "Elements of Biology” McPherson and Henderson's 
"Elements of Chemistry," Kimball's "College Physics," and 
Howell's "Physiology." 

The results below summarize the findings regarding these ma- 
terials as to (a) range of vocabulary, ( b ) size of highly techni- 
cal vocabulary (zero value words), and (c) weighted median 
index number. For each book two counts were made, in order 
to determine the reliability of the method. The second samp- 



Summary Regarding Vocabulary Burden — 16 Types of Material 



Counts 


Range 


Zero value 
words 


Weighted 

median 


1 


2 


1 


2 


1 


2 


Second readers: 


Jones 


371 


350 


4 


4 


86 


88 




Horace Mann 


412 


421 


9 


9 


78 


83 




. Aldine 


367 


353 


7 


6 


77 


79 


Fourth readers: 


Jones 


471 


454 


12 


20 


71 


62 




Aldine 


450 


455 


24 


11 


63 


69 




Horace Mann 


466 


472 


15 


17 


65 


66 


Stevenson: 


Kidnapped 


402 


415 


21 


30 


67 


65 


Thackeray: 


Vanity Fair 


490 


459 


43 


34 


43 


54 


Columbus Dispatch 


528 


581 


49 


45 


33 


37 


without local names 


514 


560 


35 


24 


38 


45 


Muzzy: 


American History 


533 


506 


24 


30 


38 


40 


Clark: 


Introduction to Science. . 


483 


491 


22 


25 


52 


50 


Clark: 


General Science 


480 


463 


30 


30 


43 


45 


Hunter: 


Elements of Biology 


464 


467 


57 


57 


28 


34 


Elementary Chemistry 


399 


358 


67 


69 


22 


14 


Kimball: 


College Physics 


393 


405 


60 


59 


24 


22 


Howell: 


Physiology 


422 


473 


108 


94 


4 


10 



sampling was made exactly as the first except that a different 
page was used. Thus, if the first count used the pages 5, 10, 

15, 20, then the second count used pages 1,6, 11, 16, the third 
line on each page being studied in each case. As will be noted, 
the method seems fairly reliable. If it is desired to increase the 
reliability it is suggested that additional thousand-word counts 
be made and the results averaged. This would seem more sat- 
isfactory than increase in the size of a single sampling since 
the thousands-word count is a very convenient unit, and after 
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a number of thousand-word counts are made, comparisons 
from one count to another are of some interest. 

III. Results 

(a) Range of Vocabulary. — The following table summarizes 
all results for these materials. As will be noted, the table 
shows (a) range, (b) zero value words, and (c) weighted me- 
dian index numbers — and for each one of these items the re- 
sults of the two counts are shown. 

The results are, however, best shown in graphic form. Chart I 
shows the difference in range. 

As might be expected, the range of vocabulary in second- 
grade readers is small. But two or three unexpected findings 
do appear. Thus the range of vocabulary in "Kidnapped" is 
also small. Stevenson evidently gets his effects not by using a 
large number of words, but by sentence structure, and other 
devices. Range in the science books is low; these books ap- 
parently use their technical terms over and over again, and use 
besides these technical terms relatively simple words. It is 
suggested that these figures may be used as tentative bases for 
comparison in further counts, the average range of the two 
samplings being employed. If the method seems of value, it is 
intended that norms for readers in the various grades, and for 
various types of books, should be developed. 

(b) Number of Zero Value Words. — Again, graphical presenta- 
tion is the most satisfactory. 

As would be expected, the second readers show the smallest 
number of words outside the 10,000 most common words. It; 
is somewhat startling, however, to find that second readers do 
include a few such terms. The number of zero value words in 
the newspaper is somewhat high. This is due in part to the 
number of local names included; the table presents also - fig- 
ures for the newspapers with these local names left out. It was 
found very difficult, however, to decide just which names 
might best be eliminated; so the chart presents the results 
without such elimination. The large number of technical trams 
in the Junior High School biology is of decided interest, and is 
indicative of the vocabulary burden of this book. The Medical 
School physiology also has a huge number of such terms, as 
would be expected. 
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(c) Weighted Median Index Numbers . — These results are pre- 
sented in graphical form in Chart III. It is felt that the 
weighted median index number is probably the best measure 
of vocabulary burden. It will be noted that "Kidnapped" is 
about at fourth grade reading difficulty; it is suggested that 
"Kidnapped" might well be used as supplementary reading at 
about the fourth or fifth grade. Other details regarding the 
comparative standing of the various books are obvious from 
the chart and need no comment. 



IV. Possible Developments Of The Method 

The question now is as to the values and limitations of the 
method as thus illustrated. But perhaps the limitations should 
be pointed out first. 
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It should be re-called in the first place that the reliability of 
these thousand-word samplings — as determined by compari- 
sons of two samplings for each book — was called “fair.” 

From trying experience in the field of tests, it has come to be 
realized that reliabilities first considered fair might be by no 
means as good as they should be. It can at least be said of the 
present study that data are presented which make possible 
some judgment as to what reliability, in a given instance, may 
be expected. More important, however, is the possibility (as 
was suggested), that the reliability may be increased as de- 
sired by taking further thousand-word samplings. Presumably, 
the reliability desired will depend upon the nicety 1 of the dis- 
tinctions which it is desired to make; very likely also the reli- 
ability of the weighted median index number and number of 
zero value words will be conditioned somewhat by the range 
of vocabulary. It is one important merit of the general proce- 
dure suggested that it is elastic, and thus adapted to such vari- 
ous demands or conditions. 

It should also be listed in the catalogue of limitations that the 
description of the sampling in terms of range, number of zero 
value words, and weighted median index number is undoubt- 
edly a description which leaves out certain important ele- 
ments. This is, of course true of any method of statistical 
summary; features appear, when the complete distribution is 
studied, which are lost in the scheme of averaging: Thus in 
the present study, the history seems to involve a greater pro- 
portional number of words in the last 5000 of the 10,000 most 
common words than any of the other books studied. But no 
special features of any of the distributions have appeared of 
sufficient prominence to demand special treatment. 

It should also be mentioned as a third handicap that the 
method is so involved in use of the Thorndike "Word Book" 
as to partake of any faults that that book may have. The writ- 
ers' work has emphasized the extent which Thorndike has 
weighted his investigation in the direction of literary and even 
poetical vocabularies. 3 The study has also made clear the ad- 
vantages which would have accrued to Thorndike's work, if a 
more systematic sampling on his part had permitted the inclu- 
sion, with the index numbers, of the frequencies with which 
each word occurred, per 1,000,000 words. The interpretation 
of median index numbers, and of other features of the total 
distributions of thousand word counts, would be much easier, 



3 Comparisons of thousand-word .counts, (not reported here) on the "Golden 
Treasury" and other literary materials with the newspaper vocabularies, as to 
overlapping, have brought out this feature. 
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and the significance of all results much enriched. 4 However, 
these are matters of relatively minor detail. The fundamental 
value of Thorndike's contribution is obvious; the "Word 
Book" has opened up a whole new field for investigation. 



Or suppose Thorndike had used for index numbers a scale of ten, for the ten 
thousand words, using decimals to distinguish the position of a word within 
each thousand. It would then be possible to read directly, from the index num- 
ber, the place of each word in the ten thousand, and interpretation would be 
greatly facilitated. 
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But now for some of the potentialities of .the method. It - 
would seem obvious in the first place that, with the develop- 
ment of such further data as might be considered to establish 
norms, such procedure should be of considerable use in 
evaluating texts or other reading material. A thousand-word 
count can be put through in about three hours. Even though 
several counts might be necessary, in order to obtain suffi- 
cient consistency from one count to another to give one confi- 
dence in the' findings, still the time cost would not appear ex- 
cessive. 

It should also be pointed out that such a systematic method of 
sampling has possibilities in investigating the distribution of 
vocabulary burden through a book. Many texts appear to have 
the vocabulary load at the beginning. A thousand-word count 
in each chapter should make possible interesting comparisons 
regarding this matter. Finally, it should be mentioned that cer- 
tain further developments of the method are possible. For in- 
stance, under certain circumstances over-lapping from one 
book to another may be of importance. Thus in studying the 
additions to technical vocabularies involved in chemistry, af- 
ter a course in general science has been taken, comparison of 
a thousand-word counts for over-lapping, from one book to 
another yields many interesting figures. : 

The reader will doubtless feel that these are great: expecta- 
tions on the basis of a few facts. Quite so it is. But the study 
has seemed distinctly suggestive; the writers are therefore 
presenting what results they have obtained, with the hope that 
others may be interested to work along these lines. 

Summary 

The paper may be briefly summarized. 

1. It is suggested that .the vocabulary difficulty or vocabulary 
burden of a book or other piece of reading material, may be 
evaluated by taking thousand-word ; samplings of the vocabu- 
laries used and examining these samplings with reference to 
the type of word employed. 

2. Three methods are suggested for summarizing the facts 
with regard to such a sampling: (a) Range of vocabulary, or 
number of different words per 1000 words sampled, (b) num- 
ber of words not occurring in the Thorndike list of the 10,000 
most common words, and (c) weighted median Thorndike 
“Word Book" index number. 
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3. Results are presented from study of, 15 books, and one 
newspaper, by the methods above indicated. 

4. It is; suggested that the general procedure has decided pos- 
sibilities, as a basis for a study of vocabulary burden. 
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1928 — Vogel and Washburne: The Winnetka Formula 
Introduction 

In 1928, Mabel Vogel and Carleton Washburne of Winnetka, 
Illinois, published one of the most important studies of read- 
ability. 1 

They were the first to study the structural characteristics of 
the text and the first to use a criterion based on an empirical 
evaluation of text. They studied ten different factors including 
kinds of sentences and prepositional phrases, as well as word 
difficulty and sentence length. Since, however, many factors 
correlated highly with one another, they chose four for their 
new formula. 

Following Lively and Pressey, they validated their formula, 
called the Winnetka formula, against 700 books that had been 
named by at least 25 out of almost 37,000 children as ones 
they had read and liked. They also had the mean reading 
scores of the children, which they used as a difficulty measure 
in developing their formula. Their new formula correlated 
highly ( r = .845) with the reading test scores. 

With this formula, investigators knew that they could objec- 
tively match the grade level of a text with the reading ability 
of the reader. The match was not perfect, but it was better 
than subjective judgments. The Winnetka formula, the first 
one to predict difficulty by grade levels, became the prototype 
of modern readability formulas. 

A Word about Correlations 

In reading research, investigators look for correlations in- 
stead of causes. A correlation coefficient (r = ) is a descrip- 
tive statistic that can go from +1 .00 to 0.0 or from 0.0 to - 
1.00. Both +1.00 and -1.00 represent a perfect correlation, 
depending on whether the elements are positively or nega- 
tively correlated. 

A coefficient of 1.00 shows that, as one element changes, the 
other element changes in the same (+) or opposite (-) direction 
by a corresponding amount. A coefficient of .00 means no 
correlation, that is, no corresponding relationship through a 
series of changes. 

For example, if a formula should predict a 9 th -grade level of 
difficulty on a 7 th -grade text, and, if at all grade levels, the er- 

1 Vogel, M. and Washburne, C. 1928. “An Objective Method of De- 
termining Grade Placement of Children’s Reading Material.” The Ele- 
mentary School Journal, Vol. 28, pp. 373-381. 
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ror is in the same direction and by a corresponding amount, 
the correlation could be +1.00 or at least quite high. If, on the 
other hand, a formula predicts a 9 th -grade level for a 6 th -grade 
text, an 8 th grade level for a 10 th -grade text, and has similar 
variability in both directions, the correlation would be very 
low, or even 0.00. 

Squaring the correlation coefficient ( r 2 = ) gives the percent- 
age of accountability for the variance. For example, the Vogel 
and Washburne formula above accounts for 71% (,845 2 ) of 
the variance of the text difficulty. 

— WHD 
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An Objective Method of Determining 
Grade Placement of Children's 
Reading Material 



MABEL VOGEL 2 

Director of Research Office, Public Schools, Winnetka, Illinois 

CARLETON WASHBURNE 

Superintendent of Schools, Winnetka, Illinois 

E very teacher has to face the problem of fitting reading 
material to children's reading ability. Any attempt in the 
past to give children suitable material has been largely a mat- 
ter of guesswork. An objective method of determining what 
material is appropriate for children of given reading ability is 
needed by classroom teachers. Similarly, in selecting text- 
books and supplementary-reading material, the superintendent 
or supervisor should have a means of knowing whether the 
books are within the reading grasp of the children for whom 
they are intended. The writers of textbooks and other books 
for children need to have an objective method of determining 
whether their vocabulary and sentence structure are such as 
will offer no serious obstacles to the children who are to read 
what they write. 

Two years ago the foundation was laid for a study of the ob- 
jectively measurable differences that exist among books read 
and enjoyed by children of various levels of reading ability. 
Thirty-six thousand seven hundred and fifty widely scattered 
children reported on all the books which they had read during 
the preceding year. The ballots which they filled out were 
brought together, and the results of their judgments make up 
the Winnetka Graded Book List. 3 

The Winnetka Graded Book List is a list of seven hundred 
books on each of which twenty-five or more children's judg- 
ments were received. This list is graded not according to the 
actual school grade of the children but according to the 
grade to which their reading ability corresponds. The para- 



2 Originally published 1928, Elementary School Journal, Vol. 28, pp. 373-381. 

3 

Carleton Washburne and Mabel Vogel, Winnetka Graded Book List, Chicago: 
American Library Association, 1926. 
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graph-meaning section of the Stanford Achievement Test 
was used as a measure of silent-reading ability. The reli- 
ability and the validity of the grading of the books in the 
Winnetka Graded Book List have been amply checked. 
Since the grade placement of these seven hundred books is 
known, it is possible to use this information in determining 
the grade placement of other books. 

One hundred and fifty-two books were chosen from the Win- 
netka Graded Book List as a basis for the present study. About 
half of them are the most popular books in the various grades. 
The other half are books well liked by both sexes and read by 
an equal number of boys and girls closely concentrated 
around the median in reading ability. This number of books 
was chosen arbitrarily as a fairly good representation of the 
books in the list. 

The Winnetka teachers’ seminar, composed of twenty volun- 
teer teachers, examined these books for every conceivable 
element of difficulty which might influence the grade place- 
ment. Examination was made of the following elements: 

1. Vocabulary difficulty (according to Pressey's technique 4 ) 

a) Number of different words occurring in a sampling of 
1,000 words 

b) Median index number (based on Thorndike's indexed 
word list 5 ) of 1,000-word sampling 

c) Number of words in 1,000- word sampling not occur- 

ring in Thorndike's list 

2. Sentence structure of seventy-five sample sentences 

a) Sentence use — declarative, exclamatory, imperative, 
and interrogative 

b) Sentence form — simple, complex, compound, and 
complex-compound 

c) Dependent clauses — noun, adjective, and adverbial 

d) Phrases — adjective, adverbial, infinitive, and particip- 
ial 

3. Parts of speech occurring in. 1,000-word sampling — 
nouns (common and proper, abstract and concrete), pro- 
nouns, verbs (action and non-action, transitive and intran- 

4 Bertha A. Lively and S. L. Pressey, "A Method for Measuring the 

“Vocabulary Burden* of Textbooks,” Educational Administration 
and Supervision, IX (October, 1923), 389-98. 

5 Edward L. Thorndike, The Teacher’s Word Book, New York: Teach- 
ers College, Columbia University, 1921. 
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sitive), infinitives, participles, gerunds, adjectives (sen- 
sory and non-sensory), articles, adverbs, prepositions, 
conjunctions, interjections, and expletives 

4. Paragraph construction 

a) Number of sentences and words per paragraph of con- 
versation and non-conversation 

b) Percentage of seventy-five sentences containing con- 
versation 

5. General structure 

a) Number of words to a line, number of lines to a book, 
and number of words to a book. 

b) Length of chapters 

6. Physical makeup 6 

a) Weight 

b) Size of type 

c) Length of line 

d) Distance between lines 

After all the elements were tabulated and counted for each 
book, each element was graphed to determine whether there 
was a definite rise or fall from grade to grade. Those elements 
showing the most definite rise or fall from grade to grade 
were chosen for further study, and the others were cast aside. 

Table I 7 shows the correlation between each of the elements 
selected for further study and the median reading score of the 
children who read the 152 books. Table II shows the intercor- 
relations of the ten most promising elements in Table I. The 
aim in choosing these ten elements was to find elements 
which would correlate as little as possible with one another 
and as highly as possible with the median reading score of the 
children who read and enjoyed the books measured. 



6 The items under this heading were not fully explored as it was felt that a dif- 
ferent technique of study was needed to determine optimum size of type, length 
of line; and leading appropriate to each grade. A separate study of these ele- 
ments is being made. 

7 

This is one of a number of similar tables constructed during the study. One 
table showed correlations with the reading grade instead of the reading score; 
another showed correlations with chronological age; etc. More than one hun- 
dred coefficients of correlation were found. The most satisfactory correlations 
were with the reading score, as shown in Table I. 
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TABLE I 

Correlation of Various Elements with Median Reading Score* 



Element Correlation 

Number of different words occurring in a sampling of 1,000 

words .770 

Median index number (based on Thorndike's list) of 1.000-word sampling — .704 

Number of words in 1,000-word sampling not occurring in Thorndike's list .. .674 



Number of words in book 

Number of phrases in 1,000- word sampling 

Number of verbs in 1.000-word sampling — 

Number of words per paragraph 

Number of prepositions in 1,000- word sampling 

Number of phrases of all kinds in 75 sample sentences 

Number of phrases and clauses of all kinds in 75 sample sentences 

Number of adverbial phrases and clauses in 75 sample sentences 

Number of adverbial phrases and clauses in 1 ,000- word sampling 

Number of adjective phrases and clauses in 75 sample sentences 

Number of adverbial phrases in 75 sample sentences 

Number of words in 75 sample sentences 

Number of simple sentences in 75 sample sentences — 

Number of conjunctions in 1,000-word sampling 

Number of adverbial clauses in 75 sample sentences 

Number of nouns in 1,000-word sampling — 

♦Because of the difference in sentence length, the number of words in seventy-five 
sample sentences varied greatly. To reduce phrase and clause counts to & common ba- 
sis in certain cases, the number of phrases or clauses was divided by the number of 
words in seventy-five sentences and the quotient multiplied by 1.000. The result 
showed the number of phrases or clauses there would be in seventy-five sentences if 
these sentences contained exactly 1.000 words. This procedure was used only when it 
yielded a better correlation than did a simple phrase or clause count of seventy-five 
sentences. It was not used in the case of any of the four elements that make up the final 
regression equation. 

Various combinations of the ten elements shown in Table II 
were tried and a series of multiple correlations found. The 
best multiple correlation (.845), combining four elements, was 
made the basis of a regression equation which predicts with a 
high degree of reliability the reading score necessary for the 
reading and understanding of any given book. The standard 
error of estimate in using this equation is 8 points on the para- 
graph-meaning section of the Standard Achievement Test. 

This means a difference of less than a grade in the lower 
grades ad a difference of slightly more than a grade in the up- 
per grades. This is a very reasonable standard error since it 
was found that any book that was read and enjoyed by chil- 
dren in a given grade could be read and enjoyed by children 
one grade above or below. 



.592 

.576 

.527 

.518 

.518 

.518 

.474 

.467 

.463 

.461 

.458 

.453 

.371 

.296 

.291 

.262 
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TABLE II 

Intercorrelation of Various Elements 





Dif- 


Prepo- 


Verbs 


Words 


Words 


Sim- 


Un- 


Ad- 


Nouns 




ferent 


sitions 


in 


per 


in 75 


pie 


com- 


verbial 


in 




Words 


in 


1,000 


Para- 


Sen- 


Sen- 


mon 


Clause 


1,000 




in 


1,000 


Words 


graph 


tences 


tences 


Words 


s in 75 


Words 




1,000 


Words 








in 75 




Sen- 




















tences 




Median reading 
score 


.770 


.518 


.527 


.518 


.453 


-.371 


.674 


.291 


-.262 


Different words in 
1,000 




.546 


-.572 


.516 


.442 


-.306 


.692 


.308 


-.177 


Prepositions in 
1 ,000 words 






-.777 


.462 


.398 


-.134 


.412 


.131 


.002 


Verbs in 1,000 
words 








-.517 


-.543 


.285 


-.431 


-.192 


.017 


Words per para- 
graph 










.706 


-.503 


.322 


.565 


-.356 


Words in 75 sen- 




















tences 












-.741 


.244 


.818 


-.399 


Simple sentences in 
75 














.106 


.674 


.552 


Uncommon words 
in 1,000 
















.074 


.069 


Adverbial clauses in 
75 sentences 


















-.424 



The elements which have a multiple correlation of .845 are as 
follows: number of different words occurring in a sampling of 
1,000 (X 2 ), number of prepositions (including duplicates) oc- 
curring in 1,000-word sampling (X 2 ), number of words (in- 
cluding duplicates) in 1,000-word sampling not occurring in 
Thorndike's list (V 4 ), and of simple sentences in 75 sample 
sentences (X 5 ). 

By making a count of these elements, any teacher can deter- 
mine the grade placement of any book. The technique used is 
as follows: 

1 . Make a sampling of 1 ,000 words from the book as fol- 
lows: 

a) Determine the number of pages in the book. 

b) Determine the number of words per line by counting 
the number of words in ten lines scattered through the 
book and dividing by 10. 

c ) Divide 1 ,000 (the number of words needed) by the 
number of words per line. For example, if there are 
eight words per line, Item c will be 1,000 divided by 
8, or 125, the number of pages from which sample 
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lines are to be chosen. 

d) Divide the number of pages in the book (for example, 
432) by the number of pages from which samples are 
to be chosen (for example, 125). In the example given, 
the quotient is 3.5. Therefore, the sample lines will be 
taken from every third page. 

e ) Copy on a separate card (cards cut 2 inches by 3 
inches are a convenient size) every word from the top 
line (or any other given line) of every page to be sam- 
pled. Put dp in the corner of each card containing a 
word used as a preposition. 

f) After copying the words from a given line on the 
number of pages estimated in c, count the cards. If 
there is not an even thousand, discard any excess, or 
add cards by copying words from additional lines until 
an exact thousand is reached. 

g) Arrange the cards in strictly alphabetical order so that 
all duplicates of any given word come together. Elimi- 
nate all duplicate cards, writing the total number of 
such cards on the one card that remains. For example, 
if there are thirty cards containing the word "the," 
write the number 30 on one “the” card and discard the 
other cards containing this word. 

2. Count the cards after the duplicates have been eliminated, 
thus obtaining the number of different words in 1 ,000. 

Call this number X 2 . 

3. Count the total number of prepositions in the 1,000 words. 
If the preposition "in," for example, occurs fifteen times, it 
should count as fifteen prepositions. Record the total 
number of prepositions as Xj, 

4. Check each word card with Thorndike's word list. Count 
the total number of words, including duplicates, which do 
not count in Thorndike's list. In this connection it must be 
remembered that derived forms of words included in the 
Thorndike list are considered as being themselves in- 
cluded in the list. For example, the word "sing" occurs In 
the Thorndike list, The word "singing" would be counted 
as being included in the Thorndike list although it will not 
be found there in this form. Thorndike's introduction to 
his word list should be carefully read to determine which 
derived farms he has not included. Record the total num- 
ber of words not included in Thorndike's list as X 4 , 

5. Make a sampling of seventy-five sentences from the book 
as follows: 
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a) Count the total number of pages in the book, exclud- 
ing picture pages. 

b) Divide the number of pages in the book by 75 to de- 
termine which pages must be chosen. For example, if 
there are 150 pages in the book, a sentence should be 
taken from every other page to make up the 75 needed 
sentences. If there are 250 pages in the book, a sen- 
tence should be taken from every third page. 

c ) Tabulate as simple or not simple the first complete 
sentence on every page to be sampled. A simple sen- 
tence is defined as one in which there are no depend- 
ent or co-ordinate clauses; it contains only one subject 
and one predicate. 

6 . Count the number of simple sentences in the 75 sentences 
sampled. Record this number as X 5 . 

7. Apply the following regression equation to the data, X\ be- 
ing the reading score, X 2 , the number of different words 
in 1,000; X 3 , the number of prepositions in 1,000 words; 
X 4 , the number of uncommon words in 1,000, and X 5 , the 
number of simple sentences in 75: 

X\ = .085X2 + .IOIX 3 + . 6 O 4 Y 4 - . 4 IIX 5 + 17.43 

The answer to the equation score will be the score on the 
paragraph-meaning section of the Stanford Achievement Test 
necessary for reading the book measured. The reading score 
may be translated into reading grade according to Table III. 

TABLE III 

Grade Standards — Paragraph-Meaning Sec- 
tion of the Stanford Achievement Test 



Score 

4-16 

18 - 34 .... 

36-52 

54-62 

64 - 70 .... 

72 - 78 .... 

80 - 86 .... 

88 - 94 .... 

96 - 102 .. 

104-112 



Grade 

II 

III 

IV 

V 

VI 

VII 

VIII 

IX 

X 

XI 
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Let us take The Japanese Empire 8 by Harry A. Franck as an 
example of the application of the equation. 

Number of different words in 1,000 = 445 

Number of prepositions in 1,000 words =117 

Number of uncommon words in 1,000 = 22 

Number of simple sentences in 75 = 20 

X, =(.085)(445) + (.101X117) + (.604X22) - (.41 1)(20) + 17.43 

The reading score necessary for the ready comprehension of 
the book is 72.14. As can be seen from Table III, this book is 
suitable for children whose reading ability is that of the aver- 
age child at the beginning of Grade VII. 

Any book for use in the elementary grades may be similarly 
analyzed. It is therefore possible to determine the correct 
grade placement for any book so far as structural difficulty is 
concerned. When books are so graded and children's reading 
ability is measured, it is possible to give children books which 
fit their ability. Furthermore, in writing a book for children in 
a given grade, an author can check his writing by the regres- 
sion equation and simplify it if necessary. For the latter pur- 
pose Table IV will be found helpful. This table gives the me- 
dians and upper and lower quartiles, grade by grade, for each 
of the four elements measured. If, through the use of the re- 
gression equation, the author finds that his material is too dif- 
ficult for the grade in which it is to be used, he can compare 
the word and sentence counts with Table IV and see which 
elements need simplification. 



8 Harry A. Franck, The Japanese Empire: A Geographical Reader. Dansville, New 
York: F. A. Owen Publishing Co., 1927. 
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TABLE IV 

Standards for Use in Writing Books for Children 





Grades 




III 


I Vs 


V 


VI 


VII 


VIII 


Number of different words in 1.000: 














Upper quartile 


338 


401 


417 


435.5 


457.5 


460.8 


Median 


316 


377 


407.5 


416.5 


440.5 


458 


Lower quartile 


258 


329 


386 


397.5 


411 


447 


Number of prepositions in 1,000 
words: 














Upper quartile 


79 


106 


110 


114 


116 


123 


Median 


71.3 


96 


100 


107 


99 


115.5 


Lower quartile 


63 


79 


83 


100 


93 


101 


Number of uncommon words in 
1,000: 














Upper quartile 


8 


14 


20 


24.5 


34.5 


40 


Median 


6 


11 


14.5 


19.5 


26 


32.5 


Lower quartile 


2 


6 


12 


17 


18.5 


28.5 


Number of simple sentences in 75: 














Upper quartile 


49 


42.5 


26 


25 


29 


34 


Median 


39 


30 


21.5 


19 


22.5 


26 


Lower quartile 


36 


22 


18 


11 


18 


21.5 



Since reading is the most basic of all school subjects and giv- 
ing children material which is too difficult in structure tends 
toward wrong methods of visual perception, lack of interest, 
and faulty understanding and is responsible for many school 
failures, an objective method of measuring the structural diffi- 
culty of reading matter for children in the elementary grades 
is of primary 

The present study deals only with structure. A similar study 
dealing with content is well under way. 
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The Early 1930s — New Directions for Readability 

Waples and Tyler: What Adults Want to Read About 

During the Depression in the ‘30s, adult education and the in- 
creased use of libraries stimulated studies in reading. Sociolo- 
gists studied “who reads what and why over consecutive peri- 
ods,” looking at reading as an aspect of mass communication. 

In 1931, Douglas Waples and Ralph W. Tyler published What 
People Want to Read About, a comprehensive, two-year study 
of adult reading interests. Instead of using the traditional li- 
brary circulation records to determine reading patterns, they 
interviewed people divided by sex and occupation into 107 
different groups. It showed the types and styles of materials 
that people not only read but also want to read. It also studied 
what they did not read and why. 

They found that the reading of many people is limited because 
of the lack of suitable material. Readers often like to expand 
their knowledge, but the reading materials in which they are 
interested are too difficult. 

Ralph Ojemann: The Difficulty of Adult Materials 

The year 1934 marked the beginning of more rigorous stan- 
dards for the formulas. Ralph Ojemann did not invent a for- 
mula, but in 1934, he did invent a method of assessing the dif- 
ficulty of materials for adult parent-education materials. His 
criterion was 16 passages of about 500 words taken from 
magazines. He was the first to use adults to establish the diffi- 
culty of his criterion. He assigned each passage the grade 
level of adult readers who were able to answer at least one- 
half of the multiple-choice questions about the passage. 

Ojemann was then able to correlate six factors of vocabulary 
difficulty and eight factors of composition and sentence struc- 
ture with the difficulty of the criterion passages. He found that 
the best vocabulary factor was the difficulty of words as 
stated in the Thorndike word list. 

Even more important was the emphasis that Ojemann put on 
the qualitative factors such as abstractness. He recommended 
using his 16 passages for comparing and judging the difficulty 
of other texts, a method that is now known as scaling (See 
“Text leveling” below). Although he was not able to express 
the qualitative variables in numeric terms, he succeeded in 
proving they could not be ignored. 
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Dale and Tyler: Adults of Limited Reading Ability 

After working with Waples, Ralph Tyler became interested in 
adults of limited reading ability. He joined with Edgar Dale to 
publish in 1934 their own readability formula and the first 
study on adult readability formulas. Dale had found problems 
with the Thorndike Word Book and started looking for better 
alternatives. The specific contribution of this study was the 
use of materials specifically designed for adults of limited 
reading ability. 

Their criterion for developing the formula was 74 selections 
on personal health taken from magazines, newspapers, text- 
books, and adaptations from children’s health textbooks. They 
determined the difficulty of the passages with multiple-choice 
questions based on the texts given to adults of limited reading 
ability. 

From the 29 factors that had been found significant for chil- 
dren’ s comprehension, they found ten that were significant for 
adults. They found that three of these factors correlated so 
highly with the other factors that they alone gave almost the 
same prediction as the combined ten. They were: 

• Number of different technical words. 

• Number of different hard non-technical words. 

• Number of indeterminate clauses. 

They combined these three factors into a formula to predict 
the proportion of adult readers of limited reading ability who 
would be able to understand the material. The formula corre- 
lated .511 with difficulty as measured by multiple-choice 
reading tests based on the 74 criterion selections. 

The Ojemann and Dale-Tyler studies mark the beginning of 
work on adult formulas that would continue unabated until the 
present time. 

Lyman Bryson: Books for the Average Reader 

During the depression of the 1930’s, the government in the 
U.S. put enormous resources into adult education. Bryson 
Lyman first became interested in non-fiction materials written 
for the average adult reader while serving as a leader in adult- 
education meetings in New York City. What he found was 
that what kept people from reading more was not lack of intel- 
ligence, but the lack of reading skills, a direct result of limited 
schooling. 

He also found out there is a tendency to judge adults by the 
education their children receive and to assume the great bulk 
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of people have been through high school. At that time, 40 to 
50 million people had a 7 th to 9 th grade education and reading 
ability. 

Writers had assumed that readers had an equal education to 
their own or at least an equal reading ability. Highly educated 
people failed to realize just how much easier it is for them to 
read than it is for an average person. They found it difficult to 
recognize difficult writing because they read so well them- 
selves. 

Although college and business courses had long promoted 
ideas expressed in a direct and lucid style, Bryson found that 
simple and clear language was rare. He said such language re- 
sults from “a discipline and artistry which few people who 
have ideas will take the trouble to achieve ... If simple writing 
were easy, many of our problems would have been solved 
long ago” (Klare and Buck, p. 58). 

Bryson helped set up the Readability Laboratory of the Co- 
lumbia University Teachers College with Charles Beard and 
M. A. Cartwright. Bryson understood that people with enough 
motivation and time could read difficult material and improve 
their reading ability. Experience, however, showed him that 
most people do not do that. 

Perhaps Bryson’ s greatest contribution was the influence he 
had on his two students, Irving Lorge and Rudolf Flesch. 

— WHD 
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1931 — Patty and Painter: The Vocabulary Burden 

Introduction 

In 1931, W. W. Patty and W. I. Painter discovered the year of 
highest vocabulary burden in high school is the sophomore 
year. 

Believing that the length of a text affects the vocabulary bur- 
den, they questioned the Lively and Pressey method of sam- 
pling 1,000-word passages from a text, 

They believed that taking a percentage of words from each 
text would give a better sample. The new method they devised 
took the words from the third line of each fifth page. 

Their formula determined the relative difficulty of textbooks 
using a combination of frequency as determined by the 
Thorndike list and vocabulary diversity (the number of differ- 
ent words in a text). 

— WHD 
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A Technique for Measuring the Vocabulary 
Burden of Textbooks 

W. W. PATTY AND W. I. PAINTER 1 
Indiana University 

A mong the important factors that should be considered 
when selecting textbook for high-school use is that of 
vocabulary burden. It seems evident that ease of reading and 
understanding the words of a textbook is an important index 
to its learning difficulty. A few studies have been made previ- 
ously in this field. 

LIMITATIONS OF SOME VOCABULARY STUDIES 

Lively and Pressey 2 selected one thousand word units from 
each of fourteen different typos of reading material. These 
words were then assigned the values given in Thorndike's 
Teachers ’ Word Book 3 and comparison was made on the basis 
of total values for each type. Their method seems to be very 
effective for the comparison of the vocabulary difficulty of 
texts or of the vocabulary burden of units equal in length fails, 
however, to consider the extra burden imposed on the reader 
by additional length. 



1 Originally published in the Journal of Educational Research, 1931, Vol. 24, 
No. 2, pp. 127-134. 

2 Lively, Bertha A. and Pressey. S. L., "A Method for Measuring the Vo- 
cabulary Burden of Textbooks," Educational Administration and Supervision, 
IX (October, 1923), 389-98 

3 Thorndike's The Teachers ’ Word Book is an alphabetical list of ten thou- 
sand words which were found to occur most widely in a count of about 625,000 
words from literature for children; about 8,000,000 words from the Bible and 
English classics; about 300,000 words from elementary school textbooks; about 
50,000 words from books about cooking, sowing, farming, the trades, and the 
like; about 90,000 words from daily newspapers and about 500,000 words from 
correspondence. Forty-one different sources were used. A measure of the fre- 
quency of each word's occurrence is given by the credit-number following it. If 
this credit number is 49 or over, it means that the word is in the first 1,000 for 
importance. A credit-number of from 29 to 48 places it in the second 1,000. A 
credit-number from 19 to 28 places it in the third 1,000. and a credit-number o£ 
14 to 18 places it in the fourth 1,000. A second column is given in the Word 
Book which indicates by number the thousand in which the first five thousand 
fall. It also indicates by the letters a and b whether the word is in the first or 
second half of that thousand. 
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Dolch 4 makes a very good point in working out a ratio 
between the number of different words to the number of total 
words in a book so as to take into consideration word repeti- 
tion. He suggests that the book having the wider range of vo- 
cabulary is certain to possess a vocabulary farther from the 
region of everyday language. This is undoubtedly true in the 
majority of cases, but it would not seem to be necessarily an 
absolute fact. 

Ward 5 took the total count of words in a section of a text and 
compared it with the Lively and Pressey method. He found a 
wider range of words in the total count and assumed that a 
thousand word count would not be a sufficient measure of any 
book. Since he has attempted to compare the results of two 
different techniques, it does not seem that he has proved his 
point. The average difficulty of words within samples would 
seem more desirable as a basis for comparing the vocabulary 
difficulty of one book with that of another than would the ex- 
treme ranges of words within the whole texts. 

It is not to be expected that, in any sampling method, we will 
arrive at results which we can set up as a fixed standard. In- 
stead, we should only expect to arrive at results on various 
texts, which, when compared, would bear the same ratio as 
would a comparison of burdens of the entire vocabularies of 
these texts. 



A SUGGESTED TECHNIQUE 

The following is suggested as a desirable technique for meas- 
uring the factor of vocabulary burden of high-school text- 
books written in tie English language. This technique was de- 
veloped and used in a research project at Indiana University, - 
in which all state-adopted texts for Indiana, with the exception 
of foreign language texts, were measured. 

LENGTH OF TEXT MUST BE CONSIDERED 

Word samples were taken from each of the texts studied. 

Since some of the books are considerably longer than others, 
it did not seem quite fair to compare their reading burden by 
selecting a definite unit of words from each book as a sample. 
The length of a book would undoubtedly affect the vocabulary 
burden of that book as compared with other books of different 
lengths. Where the difficulty of the average word is approxi- 



4 Dolch, Edward William. "Vocabulary Burden," Journal of Educational Re- 
search, XVII (March, 1928), 170. 

5 Ward, J. L. "Measuring Vocabulary Burden.” American School Board 
Journal, LXXI, page 98. 
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mately equal, it, at least, is a greater task to read a long book 
than it is to read a short one. Also, by taking a definite unit, 
we would not take into consideration the proportional repeti- 
tion of these words in any other unit similarly selected from 
the same text. 

Thus, the different words in a one-thousand-word unit from a 
short text bear a lower ratio to the total one thousand than the 
different words in the entire text might bear to the total words 
in it. In other words, the longer the text the greater is the 
probability of its having a high percentage of word repetition. 
By taking a definite percent sample from each text, the ratio 
o£ different words to total words in the sample would be more 
nearly representative of the actual ratio of different words to 
total words in the entire book. It, therefore, seems that a pro- 
portionate word sample is the only valid sampling basis for 
comparing texts of unequal length and that the results so de- 
rived are more reliable than where a definite unit is used. 

SELECTION OF THE SAMPLE 

In order to get what was considered a fair proportionate word 
sample each book, the words from the third line of each fifth 
page were listed. If this were not a full line, the next full line 
was taken except in cases where the previous five pages were 
partly given over to pictures, graphs, diagrams, etc. If this 
piece of line seemed to be proportionately comparable to the 
amount of printed matter on those pages, it was used. When 
the fifth page was given over entirely to non-printed material, 
then the next printed page was sampled; the regular order, 
however, was resumed in taking the succeeding samples. 
These words were then tabulated alphabetically and their fre- 
quency numbered as they appeared in each sample. The num- 
ber of words then in each list was the number of different 
words in each sample. This was called the range of the words. 

USING THE TEACHERS’ WORD BOOK 

Each of these different words was then looked up in 
Thorndike's Teachers ’ Word Book. The values that Thorndike 
had estimated were set down opposite each word in column 
arrangement, as is shown in the sample following. 

The figures in the column at the left indicate the frequency of 
the word as found in this word sample. The figures in the first 
column to the right indicate the word value or the Thorndike - 
index number. Where there are figures in the next column the 
first figure to the left represents the ranking of a thousand of 
the most common ten thousand words to which this particular 
word belongs; the letters a and b indicate whether it belongs to 
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the first or the second half of this thousand; and the number at 
the right, when such number is found in this column, indicates 
the quarter of that thousand into which the word falls. 



Sample Page of Word Sample Tabulation 



F. 


Word 


T.I.N. 


Position 
of in 
thousand 


W.V. 


14 


are 


181 


lal 


2534 


1 


accounts 


63 


lb 


63 


41 


and 


210 


lal 


8610 


61 


a 


208 


lal 


12688 


1 


against 


114 


lal 


114 


1 


Archimedes 


0 




0 


1 


acetylene 


0 




0 


19 


as 


204 


lal 


2652 


8 


at 


203 


lal 


1624 


2 


atmosphere 


11 


5b 


22 


5 


air 


91 


la4 


455 


1 


also 


119 


la2 


119 


8 


another 


116 


la2 


348 


1 


attached 


20 


3b 


20 


1 


apparatus 


7 




7 


2 


arrange 


35 


2b 


70 


1 


away 


125 


la2 


125 


1 


absorbed 


8 




8 


1 


aids 


47 


2a 


47 


1 


along 


99 


la3 


99 


1 


animals 


70 


lb 


70 


1 


act 


70 


lb 


70 



A WORD WEIGHTED VALUE 

In the last column we find what we have termed a weighted 
value of each word. This weighted value is a product of the 
Thorndike-index-number and the frequency of the word in 
that particular sample. We might represent this by the follow- 
ing formula: W.V. = T.I.N. X F. In this formula, W.T. repre- 
sents the weighted value; T.I.N. , the Thorndike-index-number 
and F., the frequency. The purpose of calculating such a 
weighted value is to take care of word repetition within, the 
sample, permitting each word to be considered in proportion 
to the frequency of its use. 

DIFFICULTY VARIES INVERSELY "WITH RECORDED 
VALUES 

It must be borne in mind that Thorndike's index numbers were 
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based largely on the frequency of the use of the words in- 
cluded in his Word Book; the higher the value which he places 
on a word, the more commonly that word is used in everyday 
language. The lower values found throughout this study, then, 
indicate a greater difficulty, or a greater vocabulary burden; 
that is, the difficulty or burden varies inversely with the val- 
ues recorded. 

COMPARISON OF AVERAGE WORD DIFFICULTY 

The next step undertaken was to calculate an average-word- 
weighted-value so that it would be possible to compare the 
average word difficulty of one book with that of another as 
accurately as this is possible by a sampling method. This was 
done by totaling the weighted values for all words of a sample 
and dividing the result by the total number of words in the 
sample. This might be represented by the formula: A.W.W.V. 
= T.W.V. -r T.W.S. In this formula, A.W.W.V. represents the 
average -word-related-value; T.W.V., the total weighted val- 
ues; and T.W.S. , the total words in the sample. It can be read- 
ily seen that this average-word-weighted-value is merely the 
arithmetic mean of the Thorndike-index-numbers for all - 
words of the sample. 

INCLUSION OF THE RANGE OF THE WORDS 

This average- word- weighted- value, however, does not seem 
to be quite a fair measure for the total burden of one book as 
compared with that of another, since it does not take into con- 
sideration the relation of the number of different words, or 
range, to the total number of words in the sample. We would 
riot know whether the A.W.W.Y. was the result of a few 
words used a number of times, or whether it was the result of 
a number of different words of about the same degree of diffi- 
culty used only a few times each. The reader will undoubtedly 
concede that the latter would he the greater burden, and that it 
would be especially noticeable in reading a long book of such 
proportionate range. 

In order to take this range of words into consideration, an in- 
dex number was found by dividing the average-word- 
weighted-value by the range of the words within the sample. 
This gives the ratio of the different words to the difficulty of 
the average word, as is shown in the following example: 



Text 


T.W.S 


R. 


T.W.V. 


A.W.W.V. 


I.N. 


Book I 


646 


379 


77.045 


119.26 


.315 


Book II 


1051 


433 


125.289 


119.20 


.275 
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In this example, T.W.S. equals total words in the sample; R. 
equals the range; T.W.V. equals the total weighted value; 
A.W.W.V. equals the average word weighted value; and I.N. 
equals the index number. It can be readily seen that, in the 
sample given, the A.W.W.V. is approximately equal in both 
books, but that Book II is considerably longer and has a larger 
range of words; consequently its reading burden would be 
heavier. By dividing our A.W.W.V.'s by their respective 
ranges we reduce these books, which have approximately an 
equal A.W.W.V., to index numbers which include not only 
the weighted of the words, but also the total words in the 
sample and the range of these words, varying inversely with 
the latter; that is, the larger the range the smaller is the index 
number, indicating a greater reading burden. We might de- 
velop this into a formula, as follows: 

A.W.W.V. = T.W.V. -r T.W.S. 

I.N. = (T.W.V. -r T.W.S.) A R or 

I.N. = (T.W.V. A T.W.S.) X (1 A R) or 

I.N. = T.W.V. -r (T.W.S. X R ) 

This same process should give satisfactory results in all eases. 

A 8UB -CL AS SIFIC ATION OF THE SAMPLE • 

Each sample list was next gone over, and the number of 
words found in each of the first three thousand was totaled, as 
well as was the number of those found between the third and 
tenth thousand, and of those which were not found at all in 
Thorndike's most common ten thousand words. The last group 
were called the zero value words, since there were no values 
listed for them. Each of these numbers was then divided by 
the total number of words in the sample in which it was found 
in order-to determine the percent which each was of the total. 
This can be used to compare the difficulty of the words in 
each book according to their distribution. It will also help to 
clarify the meaning of the index number which was first de- 
termined as well as to serve as a check on it 

DETERMINING THE MOST DIFFICULT YEAR OF 
SCHOOL 

Another phase of the procedure was the sending out of a ques- 
tionnaire to one hundred commissioned high schools in the 
state of Indiana. These schools were selected by taking every 
eighth commissioned high school, regardless of size, from the 
Indiana state school directory. The apparent range was from 
the smallest to the largest commissioned high schools of the 
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state. 



The final form of the questionnaire resolved itself into a check 
list containing the names of the forty-three different texts 
adopted by the state, and space for the of those used within 
the school and for checking the year or years of school in 
which each was used. Replies were received from seventy 
percent of the schools. A tabulation of the results gave us the 
frequencies of the use of each text in each year of high, 
school. By comparing this with our index number, as worked 
out in the first division of our procedure, we were able to find 
out in which year the texts having the greatest vocabulary 
burden were used. 



SAMPLE OP TEXTBOOK CHECK LIST 



Year of H.S. 



1 2 3 



Our English 

Written and Spoken 

English Literature Clippinger 

Outlines of English Lit Long 

Outlines of English Lit. readings Long 

American Literature Long 

Outlines of American Lit Long 

Outline of American Lit. Readings Long 

Early American History, Rev Webster 

Modem European History, Rev Webster 

Modem Times and Living Past Elson 

History of the U.S Beard, etc. 

Government in the U.S Smith, etc. 

Elementary Principles of Chem Brownlee, etc. 

First Book in Chem., 1928 ed Bradbury 

Chemistry and Its Uses McPherson, etc. 

Beginners Chem. and Its Uses Irwin, etc. 

Elem. Prin. of Physics Fuller, etc. 

Elements of Physics Millikan, etc. 

Essentials of Mod. Physics Dull 

Physics in Everyday Life Henderson 

Commerce and Industry Smith 

High-School Geography Whitbeck 

First Course in Algebra Nyberg 

Second Course in Algebra Nyberg 

Modem Plane Geometry Clark, etc. 

Modem Solid Geometry Clark, etc. 

Arithmetic of Business Smith 

Applied Arithmetic Smith 

Farm Projects and Problems Davis 

Animal Husbandry Harper 

Soils and Crops Mosier 

Studies in Horticulture Lloyd 

Engineering on the Farm Stewart 

Practical Botany (Agri.) Bergen 

Plant Life and Plant Uses Coulter 

Elem. Studies in Botany Coulter 

First Course in Botany Pool, etc. 

Textbook in Botany Allen, etc. 

Animal Studies Jordan, etc. 

General Zoology Linville, etc. 

Healthful Living Williams 

Practical Zoology Hegner 



Please be especially careful to check whether you are using the text or would use it, if you 
were not using an old adoption, and whether you are or would use it in years 1, 2, 3, or 4. 



4 



In order that we might more objectively compare the diffi- 



38 





The Classic Readability Studies 



1931 — Patty and Painter: The Vocabulary Burden 



culty of one year of high school with that of another, we 
worked out an average index number, or an arithmetic mean 
of the index numbers, of the total state adopted texts used in 
any one year. First, we multiplied the frequency that a text 
was used in each year by the index number for that text, deriv- 
ing what we called a frequency-index-product for that text for 
each year used. The frequency-index-products for each year, 
as well as the frequencies of the use of state adopted texts in 
each year, were then totaled. Finally these four total- 
frequency-index-products were divided by the total of the fre- 
quencies for their respective years. The result was an average- 
index-number which indicated the average vocabulary burden 
for each year of high school. We might also work this out as a 
formula, as follows: 

B.F. X I.N. equals F.I.P., in which B.F. represents the book 
frequency; I.N., the index number; and F.I.P., the frequency 
index product. 

A.I.N. equals T.F.I.P. -r N. in which A.I.N. represents the av- 
erage index number or the arithmetic mean of the index num- 
bers; T.F.I.P., the total frequency index products; and N., the 
total number of book frequencies in each year. 

It is recognized that the measurement of vocabulary burden of 
high-school textbooks is only one of several desirable devices 
for ascertaining their relative suitability for class use. The 
foregoing method is presented only as an apparent improve- 
ment in technique in one phase of measurement of the quality 
of the texts. Methods of equal or better quality should be de- 
veloped for evaluating other features that affect the worth of 
books as aids to learning. 



39 





1935 — Gray and Leary: What Makes a Book Readable 

In 1935, William S. Gray and Bernice Leary published their 
landmark work in reading research, What Makes a Book 
Readable. Like Dale and Tyler’s work, it attempted to dis- 
cover what makes a book readable for adults of limited read- 
ing ability. 

Their criterion included 48 selections of about 100 words 
each, half of them fiction, taken from the books, magazines, 
and newspapers most widely read by adults. They established 
the difficulty of these selections by a reading-comprehension 
test given to about 800 adults designed to test their ability to 
get the main idea of the passage. 

No subsequent work has examined readability so thoroughly 
or investigated so many style elements or the relationships be- 
tween them. The authors first identified 228 elements that af- 
fect readability and grouped them under these four headings: 

1 . Content 

2. Style 

3. Format 

4. Features of Organization 

The authors found that content, with a slight margin over 
style, was most important. Third in importance was format, 
and almost equal to it, “features of organization,” referring to 
the chapters, sections, headings, and paragraphs that show the 
organization of ideas (See Table I on the next page). 



They found they could not measure content, format, or or- 
ganization statistically, though many would later try (See be- 
low, “The measurement of content”). While not ignoring the 
other three causes, Gray and Leary concentrated on 80 vari- 
ables of style, 64 of which they could reliably count. They 
gave several tests to about a thousand people. Each test in- 
cluded several passages and questions to show how well the 
subjects understood them. 



1 . Content 

Propositions 

Organization 

Coherence 



4, Structure 

Chapters 

Headings 

Navigation 



tX (< 




Style 

Semantic and 

Syntactic 

Elements 



3, Design 

Typography 

Format 

Illustrations 



The four basic elements of reading ease. 
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TABLE I 



Summary or Judgment Concerning the Relative Influence on Readability 
of the Four Major Categories 



Major Category 


All Persons 


Librarians 


Publishers 


Others Inter- 
ested in Adult 
Evocation 




M. 


a 


Range 


M. 


<r 


Range 


M. 


a 


Range 


M. [ <2 


Range 


I. Format 


20. 26 


7.68 


45’5 


24.13 


7.64 


45-7 


I7.O8 


5-20 


25-1017.92! 6 27 


30-5 


Jl. (icneral Features of Or- 
ganisation 


‘5.38 


7 04 


40-3 


15.71 


6.59 


26-3 


15 -42 


6.27 


| ( 

30-5 :i5,2o| 7.24 


40-s 


III. Style of Expression and 
Presentation 


30.71 


9.17 


50-0 


32.74 


8.39 


50-2032.92 


8.27 


1 [ 

50-2527.50 9.24 


40-5 


IV. Content 


33.64 


13 I. 


75-7 


27.42 


995 


SO -7 ; 34 - 5 « 

\ 


12.83 


so-1039.37ju.54 


7 S-i° 



Per cent 2 o 40 60 80 1 00 



All Judges 
Librarians 
Publishers 
Others 




Format 



General 
features of 
organiza- 
tion 



Style of ex- 
pression and 
presentation 



Content 



Fig. 1. — Opinion concerning the influence of classified factors on readability 



The four major categories of readability (Gray and Leary, p. 31). 



Having a measure, now, of the difficulty of each passage, they 
were able to see what style variables changed as the passage 
got harder. They used correlation coefficients to show those 
relationships. 

Of the 64 countable variables related to reading difficulty, 
those with correlations of .35 or above were the following 
(P- 115 ): 

1. Average sentence length in words: -.52 (a negative corre- 
lation, that is, the longer the sentence the more difficult it 
is). 

2. Percentage of easy words: .52 (the larger the number of 
easy words the easier the material). 

3. Number of words not known to 90% of sixth-grade stu- 
dents: -.51 

4. Number of “easy” words: .5 1 

5. Number of different “hard” words: -.50 

6. Minimum syllabic sentence length: -.49 
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7. Number of explicit sentences: .48 

8. Number of first, second, and third-person pronouns: .48 

9. Maximum syllabic sentence length, -.47 

10. Average sentence length in syllables, -.47 

1 1 . Percentage of monosyllables: .43 

12. Number of sentences per paragraph: .43 

13. Percentage of different words not known to 90% of sixth- 
grade students: -.40 

14. Number of simple sentences: .39 

15. Percentage of different words: -.38 

16. Percentage of polysyllables: -.38 

17. Number of prepositional phrases: -35 

Although none of the variables studied had a higher correla- 
tion than .52, the authors knew by combining variables, they 
could reach higher levels of correlation. Because combining 
variables that were tightly related to each other did not raise 
the correlation coefficient, they needed to find which elements 
were highly predictive but not related to each other. 

Gray and Leary used five of the above variables, numbers 1, 

5, 8, 15, and 17, to create a formula, which has a correlation 
of .645 with reading-difficulty scores. An important character- 
istic of readability formulas is that one that uses more vari- 
ables may be only minutely more accurate but much more dif- 
ficult to measure and apply. Later formulas that use fewer 
variables may have higher correlations. 

Gray and Leary’s work stimulated an enormous effort to find 
the perfect formula, using different combinations of the style 
variables. In 1954, Klare and Buck listed 25 formulas for 
children and another 14 for adult readers. By 1981, Klare 
noted there were over 200 published formulas. 

Research eventually established that the two variables com- 
monly used in readability formulas-a semantic (meaning) 
measure such as difficulty of vocabulary and a syntactic (sen- 
tence structure) measure such as average sentence length-are 
the best predictors of textual difficulty. 

Some experts consider the number of morphemes for each 
100 words to be a major contributor to semantic (meaning) 
difficulty and the number of Yngve word depths (branches) in 
each sentence to be a major contributor to syntactic (sentence) 
difficulty. One study (Coleman 1971) showed that Flesch’s 
index of syllables for each 100 words correlates .95 with mor- 
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pheme counts. Another study (Bormuth 1966) found that the 
number of words in each sentence correlates .86 with counts 
of Yngve word depths. Measuring the average number of syl- 
lables per word and the number of words in each sentence is a 
much easier method and almost as accurate as measuring 
morphemes and word depths. 
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1944 — The Lorge Readability Index 

Introduction 

Irving Lorge was interested in psychological studies of lan- 
guage and human learning. At Columbia University’s Teach- 
ers College, he came under the influence of Lymon Bryson. 

In 1938, Irving Lorge published The Semantic Count of the 
570 Commonest English Words, a frequency count of the 
meaning of words rather than the words themselves. In 1944, 
he was co-author of E. L. Thorndike’s last book, The 
Teacher’s Word Book of 30,000 Words. 

Lorge wanted a simpler formula for predicting the difficulty 
of children’s hooks in terms of grade scores. 

In a 1939 article, “Predicting Reading Difficulty of Selections 
for Children,” 1 he demonstrated that new combinations of 
variables gave predictions of higher accuracy than the Gray- 
Leary formula. Lorge again established that “vocabulary load 
is the most important concomitant of difficulty.” 

In 1944, Lorge published his new Lorge Index in the Teachers 
College Record in an article entitled, “Predicting Readabil- 
ity,” reprinted here. In 1948, Lorge 2 published corrections to 
his formula, which are given here in the footnote on p. 56. 

Though created for children’s reading, Lorge’ s Index was 
soon widely used for adult material as well. Where Gray and 
Leary’s formula had five elements, Lorge’s had these three, 
setting a trend for simplifying the formulas that was to follow: 

• Average sentence length in words 

• Number of prepositional phrases per 100 words 

• Number of hard words not on the Dale list of 769 easy 
words. 

Lorge’s use of the McCall-Crabbs Standard Test Lessons in 
Reading as a criterion of difficulty greatly simplified the prob- 
lem of matching readers to texts. Although these passages 
were far from ideal, they remained the standard criteria for 
readability studies until the studies published by John Bor- 
muth of the University of Chicago in 1969. 

During and after World War II, the government bureaus and 
the Armed Services of the U.S. searched for efficient ways of 

1 Lorge, I. 1939. “Predicting Reading Difficulty of Selections for Chil- 
ren.” The Elementary English Review. Vol. 16, 229-233. 

2 Lorge, I. 1948. “The Lorge and Flesch Readability Formulae: A Cor- 
rection.” School and Society, Vol. 67, pp. 141-142. 
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assessing the readability of their materials. Lorge’ s formula 
was one of the best available, and it came into wide use. 

Lorge’ s work established the principles for the readability re- 
search that would follow and set the stage for the Dale-Chall 
and Flesch Reading Ease formulae, which were introduced in 
1948. 

— WHD 
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Vol. XLV 



March, 1944 



Predicting Readability 

IRVING LORGE 1 

Associate Professor Of Education, Teachers College 



W HAT a person understands of the mate- 
rial he reads depends upon his general 
reading ability and the readability of the text he 
is reading.. His reading ability, moreover, de- 
pends upon his intelligence, education, envi- 
ronment, and .upon his interest and purpose in 
reading. The readability of a text depends upon 
the kind and number of ideas it expresses, the 
vocabulary and its style, and upon format and 
typography. 1 

Reading .comprehension must be viewed as 
the interaction between reading ability and 
readability. Reading ability can usually be es- 
timated by a. person’s success with an ade- 
quate reading test. Readability, however, must 
be measured in terms of the success that large 
numbers of persons have in comprehending the 
text. In measuring the readability of-texts, the 
material is presented-to a random sample of 
persons whose reading, ability is known. -The 
readability of the text is assigned the average 
reading ability score of the sample. In assign- 
ing the average reading ability score as an es- 
timate of the readability of a text, one must 
assume, of course, that the variations in peo- 
ple's interests and purposes in-reading are bal- 
anced. 

The Criterion of Readability 

Research in readability originated in the desire 
to grade textbooks and other materials for use 
in the elementary grades. Subsequently, the 



1 Originally published as “Predicting Readability” 
in the Teachers College Record, Vol. 45, 404-419, 
March 1944— W.H.D. 



research activities were extended not only to 
demonstrate the lack of adequate reading mate- 
rials for adults, but also to suggest how more 
adequate materials might be prepared. The re- 
search in readability became a search for a re- 
lationship between structural elements of the 
text and some measure of success with that text 
by large groups of readers. The literature of 
readability is concerned with the criterion for 
readability as well as with predictors of read- 
ability. In terms of the definition of readability, 
the criterion must be a measure of success that 
a large number of readers would have with the 
text. Such a criterion may be obtained by 
judgment or by more objective methods of ap- 
praisal. The method of judgment utilizes rat- 
ings of estimated difficulty of texts. Recently, 
Flesch,2 using the method of judgment, as- 
sumed that the text in. magazines like The 
American Scholar, Foreign Affairs, and The 
Yale Review, was more difficult (less compre- 
hensible to a random sampling of readers) than 
the text in magazines' like True Confessions, 
Modern Screen, and. Romantic Story. 

Therefore, on the assumption that magazines 
are written on different levels of readability, he 
assigned criterion level scores to groups of 
magazines. More objective measures of read- 
ability, however, have been used, Vogel and 
Washburne's criterion for the readability of a 
book was the average paragraph meaning score 
on the Stanford Achievement Test of children 
who had read and liked that book. Gray and 



2 Studies referred to in this article, together with other 
pertinent references, are listed in the Bibliography, 
page 59. 
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Leary used the criterion of the average reading 
comprehension test score of a group of adults 
as an estimate of readability. 

Variables Used To Predict Readability 

The variables used to predict readability are 
aspects of the text, e.g., vocabulary load, sen- 
tence structure and style, and interest. One or 
more measures of vocabulary load is used as a 
predictor in every study of readability. The 
more usual measures are the following: 

(i a ) Number of running words. 

(b) Percentage of different words. 

(c) Percentage of different infrequent, un- 
common, or hard words, 

(d) Percentage of polysyllabic words. 

(e) Some weighted measure of vocabulary 
difficulty. 

(/) Vocabulary diversity (related to b). 

(g) Number of abstract words. 

(h) Number of affixed morphemes (pre- 
fixes, inflectional endings, etc.). 

Most studies also predict readability on the 
basis of one or more measures of sentence 
structure or style, e.g., 

(i) Percentage of prepositional phrases. 

(j) Percentage of indeterminate clauses. 

(k) Number of simple sentences. 

(l) Average sentence length. 

Less frequently, the prediction of readability is 
based on some measure of human interest, e.g. 

(m) Number of personal pronouns. 

(n) Number of words expressing human in- 
terest. 

(o) Percentage of colorful words. 

(p) Number of words representing funda- 
mental life experiences. 

(q) Number of words usually learned early 
in life (related to b). 

Essentially, the prediction of readability re- 
quires calculation by means of an empirical 
formula relating specific variables of readabil- 
ity to the criterion for readability. Vogel and 
Washburne developed their equation predicting 



the average grade level equivalent of the para- 
graph meaning score of those children who 
read and liked specified books from four pre- 
dictors: percentage of different uncommon 
words, number of prepositional phrases, and 
the relative number of simple sentences. 3 

Gray and Leary, after relating more than forty 
different predictors to their criterion, empiri- 
cally chose five variables to predict readability: 
the number of different words, the percentage 
of uncommon words, the relative number of 
personal pronouns, the relative number of 
prepositional phrases, and the average sentence 
length. 4 

Gray and Leary's predicted readability score 
was a number which was transmuted into, a 
letter representing areas of difficulty of read- 
ability from A (very easy) to E (very difficult): 
Lorge, basing his work on that of Gray and 
Leary, tried to obtain a prediction in terms of 
grade level of reading. The sample of materials 
chosen for analysis was the 376 passages in the 
four books of McCall and Crabbs’ Standard 
Test Lessons in Reading. The criterion was the 
grade level score, equivalent for a group of 
readers who would get half of the test ques- 
tions right on each passage. The predictors 
studied by Lorge were the five used: by Gray 
and Leary: a weighted score for- vocabulary 
based on Thorndike's 20,000 word list, and 
four elements used by' Morriss and Holversen 
(percentage of elemental words, percentage of 
simple localisms, percentage of concrete word- 
labels, and percentage of abstract word-labels). 
Later, Llesch's two factors, (affixed mor- 
phemes and human interest) were also used. 5 



3 The multiple correlations between the criterion and 
the weighted composite of the predictors was .845. 
Subsequently, Washburne and Vogel reported a multi- 
ple correlation of .869 on the basis of certain modifica- 
tions. 

4 The multiple correlations between the five predictors and 
the criterion used by Gray and Leary was .644. 

5 The multiple correlations were obtained predicting the 
criterion from various combinations of these factors. Empiri- 
cally, the best prediction using the fewest factors was ob- 
tained with three factors (also used by Gray and Leary): the 
average sentence length, the relative number of prepositional 
phrases, and the relative number of different words not com- 
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The simple three-factor prediction equation of 
Lorge's was modified by the addition of a con- 
stant to give an estimate of the grade level 
score equivalent to passing three-quarters of 
the questions on a given passage. The formula 
is given on the, work sheet, (see page 56) for 
computing the readability index. A reasonably 
good prediction of readability can be obtained 
by using a weighted composite of vocabulary 
and sentence structure. Of these, the most im- 
portant is some measure of vocabulary load. It 
should be recognized, however, that such ele- 
ments as the number of abstract words, the 
number of uncommon words, the number of 
polysyllabic words, and the weighted index of 
.difficulty of vocabulary are all intercorrelated. 
Any one of them could be used in place of any 
other, provided suitable adjustment were made 
in the empirical formula. Certainly some as- 
pect of vocabulary load must be used as a pre- 
dictor. 

Structural elements of the passage provide the 
second most important basis for estimating the 
readability of text. As in measures of vocabu- 
lary, most measures of sentence structure are 
interrelated, so that little additional information 
is yielded by several measures of sentence 
structure. 

Lorge's formula, as described in the following 
pages, uses as predictors the factor of uncom- 
mon words (vocabulary) and the factors of av- 
erage sentence length, and the relative number 
of prepositional phrases (sentence structure.) 

Formula For Judging Readability 

The Lorge formula, therefore, is a means .of 
judging the relative difficulty or readability of 
either read or spoken passages. Readability is 
based upon the comprehension of passages by 
school children. Comprehension is judged by 



mon to Dale’s list of 769 words. The multiple correlation 
coefficient between the average grade score on the 
Thorndike-McCall Reading Test and the three predictors was 
.77. Adding as predictors the weighted index for word fre- 
quency and/or the four factors of Morriss and Holversen, 
separately or in combination, and/or the two factors of 
Flesch, separately or in combination, did not increase the 
multiple correlation significantly. 



the correctness and completeness of responses 
to questions about a passage. Such questions 
usually deal with specific details, general im- 
port, appreciation, knowledge of vocabulary, 
and understanding of concepts. 

It is obvious that the purpose of the reader in 
reading and the kinds of questions asked in 
estimating reading comprehension will influ- 
ence greatly the estimate of reading difficulty. 
Since the Lorge formula is based on a criterion 
derived from responses to questions of the five 
types listed above, it tends to overestimate the 
difficulty of passages to be read primarily for 
appreciation or for general import and to un- 
derestimate the difficulty of passages to be 
read primarily for specific details or for fol- 
lowing directions. Nevertheless, the formula 
provides an overall estimate which should be 
useful in grading reading materials. As an es- 
timate, it should not be considered definitive 
nor used blindly. 

As developed in the work sheet, the readability 
index is an estimate of the reading grade at 
which the average school child will be able to 
answer with adequate completeness and cor- 
rectness about three-fourths of the questions 
concerning detail, appreciation, import, vo- 
cabulary, and concept. The reading grade so 
obtained may be thought of in terms of reading 
grade scores on a test of reading comprehen- 
sion. A readability index of 5.2 for a passage 
may be considered indicative of the material of 
the fifth grade; it may be thought in terms of 
placemen of the material as within the reading 
comprehension of average fifth grade children. 
Such placement, however, should consider the 
interest of pupils, the suitability of subject mat- 
ter, and other factors. The readability index is 
an estimate and not a rigorous determination. 

The Lorge formula, in addition to its use in 
estimating the reading difficulty of passages 
for children, may be used to advantage in esti- 
mating the difficulty of silent and oral passages 
for adults. It yields a readability index which 
places materials in relative order; that is, a 
reading passage with an index of 7.1, etc. 
Moreover, the suitability of texts for adults can 
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be interpreted in terms of the reading grade 
scores of adults on acceptable reading tests. 

Teachers of adults, or indeed, any person 
choosing tests for specific audiences, might 
give a reading test to a sample of adults to de- 
termine the average reading grade score (as 
well as the range of such scores). They then 
could choose texts within the demonstrated 
range of comprehension of such adults. 

The Readability Index 

The Lorge Readability Index, in addition to its 
utility in grading text materials, may also be 
used for passage : simplification. If the text for 
children is, let us say, designed for grade level 
6.0 and on the basis of the formula has a read- 
ing index of 7.6, then the text may be revised 
by simplifying sentence structure, by substitut- 
ing simple sentences for prepositional phrases, 
and by an adequate choice of vocabulary. 

Since vocabulary is the most important factor 
in passage difficulty, care must be taken to in- 
dicate the meaning of more difficult words by 
definition, example, or context. Choice of vo- 
cabulary, furthermore, may-be controlled by 
use of The Teacher's Word Book of 30,000 
Words, a new word book compiled by 
Thorndike and Lorge, in which every word is 
given a value according to .relative frequency 
in the English language. The value of AA indi- 
cates words that occurred a hundred or more 
times per million words; the value of A indi- 
cates words that occurred from fifty to ninety- 
nine times per million words; the values 49, 

48, 47, etc., indicate the number of times the 
word occurred-per million words, In selecting 
vocabularies for the revision of texts, a safe 
rule is to utilize, in addition to the information 
given by the index, these values recommended 
by Thorndike and. Lorge. 

In actual practice, the formula has proved, to 
be very serviceable in the simplification of 
texts for adult use, The grade placement of the 
text may be compared with the average highest 
grade reached by adults for whom it is de- 
signed. The median highest grade reached, for 
adults, twenty years and over is reported by the 



Bureau of the Census for the year 1940. For 
the adult population “20 years old and over” 
the median highest grade (number of years of 
school completed) was 8.8. In writing for such 
an average population, it may safely be as- 
sumed that the reading ability as measured by 
grade score on a reading test will be somewhat 
lower, let us say, about eight-tenths of a school 
year. Hence, in writing for a population with 
an assumed grade level score or a reading rest 
score of 8.0, steps should be taken to select 
vocabulary, simplify sentence structure, and 
reduce the number of prepositional phrases. 
Again, The Teacher’s Word. Book of 30,00 
Words should be of considerable help, since it 
gives separate evaluation for vocabularies 
found in adult magazines, e.g., Saturday Eve- 
ning Post, Ladies' Home Journal, Woman's 
Home Companion, True Story, and Reader's 
Digest. 

Computing the Readability Index 

The following are directions for computing the 
readability index. 

A. Selecting the sample: 

1 . Short passages of 100 words or less. 

When a short passage is to be ap- 
praised, it is advisable to analyze the 
entire passage. 

2. Longer passages. 

When longer passages are to be ap- 
praised, it is advisable to analyze sam- 
ples of the material. Select a sample 
near the beginning, another sample, 
near the middle, and another sample 
near the end of the passage. Each of 
these samples should be approximately 
one hundred words in length. 

A good procedure might be to number 
the lines of text serially and then count 
the number of words per line (about ten 
lines) to get an estimate of the number 
of words. For instance, a passage has 
141 lines; ten lines chosen at random 
have 11, 12, 13, 13, 12, 12, 12, 12, 16, 
and 16 words, or an average of 13 
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words to the line. The passage thus has 
approximately 1,833 words. A sample 
of 100 words would then be approxi- 
mately eight lines in length. The three 
samples could be chosen in a variety of 
ways: e.g., beginning at or near line 3 
through line 1 1; at or near line 53 
through line 61; and at or near line 103 
through line 1 1 1 . In this way, a sample 
is chosen in each third of the passage. 

It should be noted, moreover, that each 
sample should start with the beginning 
of a sentence and should stop at the 
end of a sentence. When, the samples 
have been located with beginning and 
end points, the remainder of the analy- 
sis can be made. 

3. Books 

When books are to be appraised, it 
would be advisable to analyze samples 
of the book, say, from 5 per cent to 10 
per cent of the book (but never less 
than five samples). These samples 
should be chosen throughout the book. 
For instance, a book has 92 pages of 
text with an average of 195 words per 
page. This indicates an approximate 
wordage of 18,000 words. A 5 per cent 
sample would be 900 words; a 10 per 
cent sample would be 1,800 words. For 
the 5 per cent sample this would re- 
quire approximately five pages; for the 
10 per cent sample, approximately nine 
pages. Thus every eighteenth page 
should be chosen for the 5 per cent 
sample; every tenth page, for the 10 
per cent sample. Thus the sample 
might be pages 3, 21, 39, 57, 75 in the 
one instance; or 4, 14, 24, 34, 44, 54, 
64, 74, 84 in the other. Of course, a 
sample must start with the beginning of 
a sentence and stop at the end of a sen- 
tence. 

B. Labeling the work sheet. 

1 . Fill out the information about the title, 
author, edition, publisher, and date, of 
publication (latest copyright year 



listed). 

2. Carefully identify the location of the 
sample, thus: “p. 14, line 2, The an- 
swer..^. 14 line 26, ever after.” 

C. Counting the number of words. 

1 . Begin with the beginning of the sample 
and count (or number serially) each 
word in the sample. Observe the fol- 
lowing rules: 

(a) Hyphenated words are counted as 
one word. When in doubt about un- 
common hyphenations, follow 
Webster’s Unabridged Dictionary 
(2nd edition); if listed in dictionary 
as hyphenated, count as one word; if 
not listed, count as two words. 

(b) Words separated at the end of a line 
to the beginning of the next line are 
counted as one word. 

(c) Numbers are counted as words, e.g., 
in “January 3, 1940” 3 is counted as 
one word and interpreted as the 
word three, 1940 is counted as one 
word and interpreted as nineteen- 
forty. 

(d) Compound words like place names 
or persons’ names are counted as 
one word, e.g., New York, United 
States, van Loon, Santa Claus, St. 
Nicholas. 

(e) Contractions are counted as one 
word; e.g., don’t, he’s, they'll, 
they’d, etc., are each counted as one 
word. 

2. Record the count under Basic Data, 
number 1. 

D. Counting the number of sentences. 

1 . Begin at the beginning of the sample 
and count the number of complete sen- 
tences. 

2. Record the count under Basic Data, 
number 2. 
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E. Counting Prepositional Phrases. 

1 . Count each prepositional phrase in the 
sample. Observe the following rules: 

(f) A phrase is made up of a preposition 
and a noun, or a preposition and a 
pronoun, or a preposition and a ger- 
und, e.g., to the house (noun), for 
him (pronoun), in skating (gerund). 

(g) Some common prepositions are: 



about 


from 


above 


in 


across 


inside 


after 


into 


along 


of 


among 


off- 


at 


on 


before 


onto 


behind 


outside 


below 


till 


beneath 


to 


beside 


under 


beyond 


up 


by 


upon 


during 


with 


except 


within 


for 


without 



(. h ) Less common prepositions are: 
despite (the opinion), concerning 
(the idea), notwithstanding (the op- 
position). 

(0 Infinitive phrases are not to be 
counted. An infinitive phrase is 
made up of the word to and a verb, 
e.g., to swim, to sing, to answer. 

( j ) If a preposition word is followed by 
a clause, it is a conjunction, and 
hence is not counted, e.g., “ After the 
storm had passed” is not counted. 

2. Record the count under Basic Data, 
number 3. 



F. Counting hard words. 

1 . Use the Dale list 6 to cross out in the 
sample every word on the Dale list, re- 
gardless of its meaning. 7 The list is 
given on pages 56 to 59. 

2. Since the count is the number of differ- 
ent hard words, each hard word is 
counted only once. For instance, if in 
the passage reliability occurred three 
times, it still would be counted only 
once. 

Observe the following rules: 

(k) Nouns. 

Separate counts are not made of plu- 
rals and possessives in s, plurals in 
es, or plurals in which y is replaced 
by ies: e.g., boys, churches, berries 
are counted with boy, church, berry, 
however, knife and knives, goose 
and geese, man and men are all 
counted as different words. 

(/) Special cases. 

An s added to a word in the text not 
forming a plural or possessive forms 
a different word from the root form: 
e.g., Robert and Roberts are two dif- 
ferent words. 

Proper nouns which seem to be 
composed of root and derived forms 
are not tabulated with the root form: 
e.g., Wheeling, the proper name is 
not counted with wheel. Browning, 
the proper name, is not counted with 
brown. Nouns formed by adding r 
or er to the other nouns or to verbs 
are not counted with the original 
word: e.g., own and owner are two 
different words. 

(to) Adverbs. 

Separate counts are not made of ad- 
verbs formed by adding ly: e.g., 

6 The list is reproduced by permission of the author. 
Dr. Edgar Dale. 

7 That is, spring, meaning season, jump, water, or 
steel coil, is counted as one word. 
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badly, sadly are counted with bad, 
sad. 

Adverbs formed from an adjective 
ending in e, as gently from gentle, 
truly from true, are counted as dif- 
ferent words. 

( 7 ?) Adjectives 

Separate counts are not made of ad- 
jectives formed by adding n to 
proper nouns: e.g., Austrian, Bavar- 
ian are counted with Austria, Bava- 
ria. 

(o) Special cases. 

An adjective formed by adding ly to 
a noun is counted as a different 
word from the noun: e.g., home and 
homely are two different words. 

(p) Comparatives and superlatives of 
adjectives and adverbs. 

Special counts are not made of 
comparatives and superlatives 
formed by adding er or r and est or 
by changing^ to ier and iest: e.g., 
longer, prettier, bravest are counted 
with long, pretty, brave. 

{q) Special cases. 

The rule apples to adjectives dou- 
bling the final consonant and adding 
er and est: e.g., red, redder, reddest 
are counted as one word. 

( 7 -) Verbs. 

Special counts are not made of verb 
forms ending in ing and in s, d, ed, 
or of forms changing y to ies and ied 
or of past participles formed by add- 
ing 77 : e.g., plays, playing, played 
are counted with play. 

(, s ) Special cases. 

Verb forms which drop the final e 
and add ing are counted with the 
root form: e.g., pace and pacing are 
counted as one word. 

Verb forms which double the final 
consonant and add ing or ed are 
counted as one word: e.g., drip, 



dripped, and dripping are counted 
as one word. 

Past participles formed by adding en 
to a verb are counted as different 
from the verb: e.g., eat and eaten 
are two different words. 

(t) Hyphenated words. 

In case of uncommon hyphenated 
words, follow Webster’s Un- 
abridged Dictionary (2nd edition). 
Any hyphenated word is considered 
as one word if it is listed thus in the 
dictionary; otherwise it is counted as 
two words. 

(u) Compound words. 

Compound names of persons or 
places, like New York, United 
States, St. Louis, Santa Claus, and 
Van Dyke, count as single words. 

(v) Contractions 

Count contractions as different 
words from those from which they 
are derived: e.g., because and 
‘cause are two different words. He ’s 
is not counted with he or with is. 

(w) Words which may be both common 
and proper. 

In the case of words which may be 
both common and proper nouns, 
count the proper noun as being the 
same word as the common: e.g., 
Jack and jack are the same word. 

(x) Miscellaneous special cases. 

Words formed by adding >> to a 
word in the list are counted as dif- 
ferent from the root word: e.g., snow 
and snowy are different words. 
German and Germany are different 
words. 

Words of different spelling listed in 
the dictionary as one word are 
counted as the same word: e.g., 
honor and honour are the same 
word. Frankfort and Frankfurt are 
the same word. 
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If a word is formed by adding two 
or more suffixes to a listed word, 
one of which when added to the 
listed word is counted with it, that 
word is different from the root 
word: e.g., happen and happening 
are the same word but happenings is 
a different word. Excite and excited 
are the same word, but excitedly is a 
different word. 

Words formed by adding en are 
counted as different from the origi- 
nal word: e.g., wool and woolen are 
two different words, bit and bitten 
are two different words. 

3. Record the count under Basic Data, 
number 4. 

G. Proceed to computation. Watch decimal 
points carefully. Check all computa- 
tions. 

H. Record on the work sheet the index (R. 

I.) to one decimal place. 

I Make sure that the analyst, computer, 
and checker have signed the record 
blank and dated their entries. 

J. If a book or a long passage has had sev- 
eral samples selected from it, the aver- 
age of the R. I.’s is the rating for the 
passage of the book. 

The Dale list of easy words is made up of 
words which are common to Thorndike’s first 
thousand most frequent English words and the 
first thousand most frequent words known by 
children entering the first grade. It is a list of 
words that are likely to be known by all chil- 
dren and adults. The Dale list, therefore, can be 
used to estimate ease of vocabulary; or, if the 
easy words are eliminated, an estimate of vo- 
cabulary difficulty can be made. 

The passage chosen to illustrate the mechanics 
of estimating the readability index is the first 
revision of the Gettysburg Address. 



Lincoln's Gettysburg Address 

Four score and seven years ago our fathers 
brought forth on this continent a new nation, con- 
ceived in Liberty and dedicated to the proposition that 
all men are created equal. Now we are engaged in a 
great civil war, testing whether that nation, or any 
nation so conceived and so dedicated, can long en- 
dure. We are met on a great battlefield of that war. 

We have come to dedicate a portion of that field, as a 
final resting place for those who here gave their lives, 
that that nation might live. It is altogether fitting and 
proper that we should do this. But, in a larger sense, 
we cannot dedicate — we cannot consecrate — we can- 
not hallow — this ground. The brave men, living and 
dead, who struggled here, have consecrated it, far 
above our poor power to add or detract. The world 
will little note, nor long remember what we say here, 
but it can never forget what they did here. It is for us, 
the living, rather to be dedicated here to the unfin- 
ished work which they who fought here, have thus far, 
so nobly advanced. It is rather for us to be here dedi- 
cated to the great task remaining before us — that from 
these honored dead we take increased devotion to that 
cause for which they here gave the last full measure of 
devotion — that we here highly resolve that these dead 
shall not have died in vain — that this nation, under 
God, shall have a new birth of freedom — and that, 
government of the people, by the people, for the peo- 
ple, shall not perish from the earth. 

The usual procedure is to cross out all words 
on the Dale list, to encircle all prepositions, 
then to list the words not on the Dale list. An 
example of the listing of the hard words is 
given on the next page. 
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Illustration of Listing of Hard Words 



A 


altogether 


add 


advanced 


B 


battlefield 


birth 




C 


continent 


conceived / 


created 


D 


dedicated ///// 


detract 


devotion / 


E 


equal 


engaged 


endure 


F 


forth 


final 


fought 


G 


government 






H 


hallow 


honored 




I 


increased 






J 








K 








L 


Liberty 






M 








N 


nation //// 


nobly 




O 








P 


proposition 


portion 


proper 


Q 








R 


remaining 


resolve 




S 


score 


sense 


struggled 


T 


testing 


thus 


task 


U 


unfinished 






V 


vain 







WXYZ 

After the number of sentences has been 
counted, the work sheet may be completed 
as shown on page 56. 

The Lorge, Readability Index was devel- 
oped after an analysis of the relationship 
between the score of readability for each 
of 376 passages and three internal meas- 
ures of vocabulary and sentence structure. 
The resulting formula predicts .readability 
well. 

Teachers will find the directions for apply- 
ing the formula simple and direct. The 
time required to analyze a passage is rela- 
tively 



3 

2 

civil cannot // consecrate / 6 

3 

3 

freedom 4 

1 

2 

1 



1 

2 

power perish 5 

2 

3 

3 

1 

1 

43 

short. Teachers will find that the expendi- 
ture of time and effort in grading materials 
is easily justified in terms of the increased 
understanding of, and the possible reduc- 
tion of, difficulties in communication. 

The meaning of the index is simply the 
school grade at which the passage can be 
understood. The index, therefore, can be 
used to place texts and other books in ap- 
propriate grades: further, it should indicate 
ways in which passages may be rewritten 
to be appropriately placed for designated 
readers. 



55 




1944 — The Lorge Readability Index 



The Classic Readability Studies 



Formula for Estimating Grade Placement of Reading Material 



Work Sheet 

Title of article: Gettysburg Address Edition: first revision 

Name of author: Abraham Lincoln 

Publisher: Date of Publication: Nov. 19, 1863 

Location of sample in text: Complete R. I. = 6.5 

Basic Data 

1 . The number of words in the sample 269 

2. The number of sentences in the sample 10 

3. The number of prepositional phrases in the sample 26 

4. The number of hard words in the sample 43 

Computation 28 



Item 6, average sentence length: 


Divide 1 by 2 = 26.90 x 


.07 = 1.8830 


Item 8, ratio of propositional phrases: 


Divide 3 by 1 = .0967 x 


13.01 = 1.2581 


Item 9, ratio of hard words: 


Divide 4 by 1 = .1599 x 


10.73 = 1.7151 




Constant = 1.6126 




Add 6, 8, 9, and C 





Readability Index: 6.4694 



Notes 

lives, n. called easy 

Last sentence, although long, is broken up by adequate punctuation 



Name of Analyst: I. D. L 
Name of Computer: I. D. L. 
Name of checker: J. C. 



Date of analysis: Nov 23, 1943 
Date of computing: Nov. 23, 1943 



The Dale List of 769 Easy Words 



A 


along 


around 


bear 


best 


bone 


a 


already 


as 


beat 


better 


book 


about 


also 


ask 


beautiful 


between 


bom 


above 


always 


at 


because 


big 


both 


across 


am 


away 


bed 


bill 


bottom 


act 


American 




bee 


bird 


bow 


afraid 


an 


B 


been 


bit 


box 


after 


and 


baby 


before 


black 


boy 


afternoon 


animal 


back 


began 


bless 


branch 


again 


another 


bad 


begin 


blind 


brave 


against 


answer 


bag 


behind 


blood 


bread 


ago 


any 


ball 


being 


blow 


break 


air 


anything 


band 


believe 


blue 


breakfast 


all 


apple 


bank 


bell 


board 


bridge 


almost 


are 


basket 


belong 


boat 


bright 


alone 


arm 


be 


beside 


body 


bring 



28 1948 Lorge corrections: In Item 6, change “ x .07” to “ x .06” 
In Item 8, change “ x 13.01” to “ x .10” 

In Item 9, change “ x 10.37” to “ x .10”. 

and for the Constant, change “ = 1.6126” to “ = 1.99” — WHD 
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broken 


coal 


drink 


brother 


coat 


drive 


brought 


cold 


drop 


brown 


color 


dry 


building 


coming 




built 


company 


E 


burn 


cook 


each 


busy 


cool 


ear 


but 


com 


early 


butter 


comer 


earth 


buy 


cost 


east 


by 


could 


easy 




count 


eat 


C 


country 


edge 


cake 


course 


egg 


call 


cover 


eight 


came 


cow 


either 


can 


cried 


else 


cap 


cross 


end 


captain 


crowd 


England 


car 


crown 


English 


care 


cry 


enough 


careful 


cap 


even 


carry 


cut 


evening 


case 




ever 


catch 


D 


every 


cause 


dance 


everything 


center 


day 


expect 


chair 


dead 


eye 


chance 


dear 




change 


deep 


F 


chief 


did 


face 


child 


die 


fair 


children 


different 


fall 


choose 


dinner 


family 


Christmas 


do 


fancy 


church 


doctor 


far 


circle 


does 


farm 


city 


dog 


farmer 


class 


done 


fast 


clean 


don’t 


fat 


clear 


door 


father 


clock 


double 


feed 


close 


down 


feel 


cloth 


draw 


feet 


clothes 


dream 


fell 


cloud 


dress 


fellow 



felt 


golden 


how 


fence 


gone 


hundred 


few 


good 


hunt 


field 


got 


hurry 


fill 


grass 




find 


gray 


I 


fine 


great 


I 


finger 


green 


ice 


finish 


grew 


if 


fire 


ground 


in 


first 


grow 


Indian 


fish 


guess 


instead 


fit 




into 


five 


H 


iron 


fix 


had 


is 


floor 


hair 


it 


flower 


half 


its 


fly 


hall 




follow 


hand 


J 


food 


hang 


jump 


foot 


happy 


just 


for 


hard 


K 


forget 


has 


keep 


fourth 


hat 


kept 


found 


have 


kill 


four 


he 


kind 


fresh 


head 


king 


from 


heard 


knee 


front 


heart 


knew 


fruit 


heavy 


know 


full 


help 






her 


L 


G 


here 


lady 


game 


herself 


laid 


garden 


hide 


lake 


gate 


high 


land 


gave 


hill 


large 


get 


him 


last 


gift 


himself 


late 


girl 


his 


laugh 


give 


hold 


lay 


glad 


hole 


lead 


glass 


home 


learn 


go 


hope 


leave 


God 


horse 


left 


going 


hot 


leg 


gold 


house 


lesson 
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let 


month 


off 


letter 


moon 


office 


lie 


more 


often 


lift 


morning 


old 


light 


most 


on 


like 


mother 


once 


line 


mountain 


one 


lion 


mouth 


only 


lips 


move 


open 


listen 


Mr. 


or 


little 


Mrs. 


other 


live 


much 


out 


lead 


music 


out 


long 


must 


outside 


look 


my 


over 


lost 

lot 


myself 


own 


loud 


N 


P 


love 


name 


page 


low 


near 


paint 




neck 


pair 


M 


need 


paper 


made 


neighbor 


part 


mail 


neither 


party 


make 


nest 


pass 


man 


never 


path 


many 


new 


pay 


march 


New York 


pen 


mark 


next 


people 


market 


nice 


pick 


matter 


night 


picture 


may 


nine 


piece 


me 


no 


place 


mean 


noise 


plain 


measure 


none 


plan 


meat 


noon 


play 


meet 


nor 


please 


men 


north 


point 


met 


nose 


poor 


middle 


not 


post 


might 


note 


pound 


mile 


nothing 


present 


milk 


now 


press 


mill 


number 


pretty 


mind 




pul 


mine 


0 


put 


minute 


oak 




miss 


ocean 


Q 


money 


of 


quarter 



queen 


seed 


something 


quick 


seem 


sometime 


quiet 


seen 


song 


quite 


self 


soon 




sell 


sound 


R 


send 


south 


race 


sent 


space 


rain 


serve 


speak 


ran 


set 


spot 


rather 


seven 


spread 


reach 


several 


spring 


read 


shake 


square 


ready 


shall 


stand 


real 


shape 


star 


reason 


she 


start 


red 


sheep 


station 


remember 


shine 


stay 


rest 


ship 


step 


rich 


shoe 


stick 


ride 


shop 


still 


right 


short 


stone 


ring 


should 


stood 


river 


shoulder 


stop 


road 


show 


store 


rock 


shut 


storm 


roll 


sick 


story 


roof 


side 


straight 


room 


sign 


street 


rose 


silk 


strike 


round 


silver 


strong 


row 


sing 


such 


run 


sir 


sugar 




sister 


suit 


S 


sit 


summer 


said 


six 


sun 


sail 


size 


suppose 


salt 


skin 


sure 


same 


sky 


surprise 


sand 


sleep 


sweet 


sat 


slow 




save 


small 


T 


saw 


smile 


table 


say 


smoke 


tail 


school 


snow 


take 


sea 


so 


talk 


season 


soft 


tall 


seat 


sold 


taste 


second 


soldier 


teach 


see 


some 


teacher 
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tear 


three 


try 


tell 


through 


turn 


ten 


throw 


twelve 


than 


tie 


twenty 


thank 


till 


two 


that 


time 




the 


tire (d) 


U 


their 


to 


uncle 


them 


today 


under 


then 


together 


until 


there 


told 


up 


these 


tomorrow 


upon 


they 


tongue 


us 


thick 


too 


use 


thin 


took 




thing 


top 


V 


think 


touch 


valley 


this 


town 


very 


those 


trade 


visit 


though 


train 




thought 


tree 


W 


thousand 


true 


wait 



walk 


wheel 


without 


wall 


when 


women 


want 


where 


wonder 


war 


whether 


would 


warn 


which 


word 


was 


while 


work 


wash 


white 


world 


waste 


who 


would 


watch 


whole 


write 


water 


whom 


wrong 


wave 


whose 




way 


why 




we 


wide 


Y 


wear 


wild 


yard 


weather 


will 


year 


week 


win 


yellow 


well 


wind 


yes 


went 


window 


yesterday 


were 


wing 


yet 


west 


winter 


you 


what 


wish 


young 


wheat 


with 


your 
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1948 — The Dale-Chall Readability Formula 
Introduction 



The Dale-Chall readability formula arrived at the end of a na- 
tional research effort that began in the 1920s and that eventu- 
ally resulted in over a thousand published studies on the read- 
ability formulas. 



In the beginning, the purpose was to develop reading materi- 
als for first-generation immigrants coming into high school. 

This research intensified in the Second World War as the U.S. 
war effort focused on the need for clear and expressive writ- 
ing. After the war, scholars harvested those hard-won lessons, 
and they gave us a new set of readability formulas for creating 
written materials for adults with limited reading ability. 

The formulas created at that time, including the Dale-Chall 
formula, the Flesch Reading Ease formula, and the Gunning 
Fog Index, have remained the workhorses of many sectors of 
commerce, education, the military, and government. 




Edgar Dale, a leading 
figure in communications, 
stressed the importance of 
vocabulary in assessing 
readability 



Of all the readability formulas, the Dale-Chall formula has 
consistently been the most reliable. It has a correlation coeffi- 
cient of .92 with comprehension as measured by reading tests. 
Most of the readability formulas use a word variable and a 
sentence-length variable, Unlike most other modern formulas, 
the Dale-Chall formula uses a list of 3,000 easy words. Using 
the formula requires counting the number of “hard” words — 
those not on the list. Doing this manually becomes easy with 
practice. There are also a few computer programs available 
online that apply the formula for you. 

For 25 years a professor of education at Ohio State Univer- 
sity, Edgar Dale was a respected authority on communica- 
tions. He worked his whole life to improve the readability of 
books, pamphlets, and newsletters — the stuff of everyday 
reading. 

Dale was one of the first critics of the Thorndike vocabulary 
lists. He claimed it failed to measure the familiarity of words 
accurately. He subsequently developed new lists that were 
later used in readability formulas. Of major importance was 
The Living Word Vocabulary: A National Vocabulary Inven- 
tory, which he wrote with Joseph O’Rourke. This work, pub- 
lished by the publishers of World Book Encyclopedia in 1981, 
lists the grade levels of 40,000 words. 



In 1948, Dale published the formula he developed with 
Jeanne Chall. She later was the founder and director for 20 
years of the Harvard Reading Laboratory. She also led the 
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Jeanne S. Chall 
created the Harvard 



Reading Lab and 
directed it for 20 
years. 



battle for teaching early reading systematically with phonics. 
Her 1967 book Learning to Read: The Great Debate , brought 
research to the forefront of the debate. For many years, she 
also was the reading consultant for TV’s Sesame Street and 
The Electric Company. 

Dale and Chall introduced their readability formula in two in 
two issues of the Educational Research Bulletin. They in- 
cluded this simple disclaimer, “We do not claim the formula 
developed here is definitive. The nature of the multiple - 
correlation coefficient makes this point rather obvious. We do 
believe, however, that it is a short cut in judging the difficulty 
of written materials.” 

Millions of readers, young and old, throughout the world have 
benefited immensely from the work of Edgar Dale and Jeanne 
S. Chall. No small measure of these benefits has resulted from 
the use of their easy-to-use and reliable readability formula. 

— WHD 
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A Formula for Predicting Readability 

By EDGAR DALE and JEANNE S.CHALL 

S everal months ago the editor of the Wall Street Journal 
ran a full-page advertisement, in one of the leading liter- 
ary magazines, announcing two honors recently awarded to it. 
One of these honors was a statement made by Robert P. Gun- 
ning that the Wall Street Journal had "the most readable front 
page in the country." 1 

How did Mr. Gunning come to this conclusion? Did he actu- 
ally sample a cross section of the readers, have them read the 
front pages of leading, newspapers, and then compare their 
ability to read and understand the various front pages? No. He 
used an accepted short cut. He predicted the reading difficulty 
of the various front pages by using a readability formula and 
found that the Wall Street Journal was the "most readable.” 

This recognition by leading journalists that readability is an 
important selling point for their newspapers is an event that is 
still quite new on the publishing horizon. Although some ob- 
jective techniques for measuring readability have been known 
for at least twenty-five years, they have been neatly buried in 
educational and psychological journals, doctoral dissertations, 
Masters’ theses, and the like. If the techniques were used at 
all, they were confined to children's textbooks. What has 
taken the dust off the technical journals and made readability 
a household word in the writing and publishing field? 

As in the rise in popularity of any technique, there was a 
critical need for this one. The war period made us realize 
more than ever the importance of reaching large audi- 
ences. More, people had to fill out tax forms; more people 
had to be appealed to to buy war bonds; more people had 
to co-operate in numerous activities to help win the war. 
Because a larger audience had to be reached, the writers had 
to use a style that could be understood by more persons than 
the usual book readers. They could no longer afford to hit or 
miss with printed materials. 



1 Originally published January 21, 1948 in Educational Research Bulletin , Vol. 
27, No. 1, pp. 11-20, 28. 



63 






